The best dashboards are built declarative with Markdown and SQL

[rinie]

I was asked to keep this viewpoint separate from issues
#1346
#1405

Also observing the rise of DuckDb and Motherduck convince me that
'The best dashboards are built with code.' does not scale.
SQL/DuckDB versus Panda's or R enables colleagues and users that are fine using Excel adjusting 90% of a dashboard themselves.

For the remaining 10% I can help optimizing their SQL or use some trick like Hive Partitioning.
Promoting the 'any dataloader' path, reduces Observable Framework to a tool for developers.

DuckDB videos underline removing the barriers of entry that a Spark Cluster or even a Postgress DB has.
Just as Duck strives to 'just work' for CSV or Parquet instead of telling the users that their input is not valid.
IMHO you should target the modern browser stack + DuckDB WASM or as an implicit dataloader for CSV or SQL/Parquet.

Python/R etc can have their uses but just like Quarto it seems more valuable to 'just works with you less then perfect data in CSV, Excel or Parquet' and incremental data updates then 'you have to transform your data or write a Python script to start'

I can always optimize the SQL to perform better, non developers cannot improve a Python script...

[jaanli]

I was asked to keep this viewpoint separate from issues #1346 #1405

Also observing the rise of DuckDb and Motherduck convince me that 'The best dashboards are built with code.' does not scale.

I can always optimize the SQL to perform better, non developers cannot improve a Python script...

Strongly disagree with this - @onefact we have taught high school students, college students, postgraduate (Master's and PhD) and faculty in non-technical subjects (e.g. humanities) to improve python scripts.

(And to write SQL; the homework from the first week of class last summer - colab here - was featured in the @motherduckdb blog: https://motherduck.com/blog/introducing-column-explorer/ and is currently used in the onboarding user flow as an example dataset.)

Please let me know if you need further examples - if we are wrong about this assumption or our experience we need to revise our data thinking curricula for the fall semester!

[rinie]

Sorry but my experience with only using YAML is that mere mortals don't grasp significant whitespace.
Which I also hate about python.
DuckDB https://duckdb.org/2022/05/04/friendlier-sql.html
DuckDB founders not specifically DBMS and Spark but I think YAML en Python have the same issues:

https://www.youtube.com/watch?v=9OFzOvV-to4&ab_channel=PositPBC

[jaanli]

I agree!

Sorry but my experience with only using YAML is that mere mortals don't grasp significant whitespace. Which I also hate about python. DuckDB https://duckdb.org/2022/05/04/friendlier-sql.html DuckDB founders not specifically DBMS and Spark but I think YAML en Python have the same issues:

https://www.youtube.com/watch?v=9OFzOvV-to4&ab_channel=PositPBC

😂😂 love that diagram of the hamburger!

I also hate YAML with a passion, I think there are better tools like pkl from @apple and https://github.com/rails/jbuilder and python data classes that we have used to replace most YAML in our stack @onefact.

I think dbt has also really helped us move away from YAML, because it let us version control hospital prices from thousands of hospitals via duckdb SQL queries.

Always happy to chat / discuss - and let me know if you’ve found better YAML replacements :) ideally statically type checked and hinted!