restructure & expand docs #1187

mbostock · 2024-04-02T03:31:11Z

This…

Adds a new What is Framework? introduction
Renames Routing to Projects
Folds CSS: Card into Markdown
Folds CSS: Grid into Markdown
Folds CSS: Note into Markdown
Folds JavaScript: Display into JavaScript
Moves JavaScript: Reactivity to Reactivity
Folds JavaScript: Generators into Reactivity
Folds JavaScript: Promises into Reactivity
Folds JavaScript: Mutables into Reactivity
Folds JavaScript: Inputs into Reactivity
Moves JavaScript: Imports to Imports
Moves JavaScript: Files to Files
Keeps JavaScript, Reactivity, and Imports together (code)
Then keeps Data loaders, Files, and SQL together (data)
Folds CSS: Color into Themes
Inverts the example thumbnails to make them stand out (light-on-dark or vice versa)
Replaces the chess example with the mortgage rates example
… and lots of other edits throughout

The overall idea is to have fewer pages, and to give the key concepts more top-level prominence.

Fixes #932.
Fixes #880.

mbostock · 2024-04-05T05:17:22Z

I’m making progress. Added a Projects page. Rewrote the Reactivity page. I think the Data page is the big remaining piece now.

Fil

So much better! I just have a few suggestions.

docs/getting-started.md

docs/what-is-framework.md

Fil · 2024-04-07T13:43:14Z

docs/what-is-framework.md

+
+## The power of code
+
+Good data apps are highly customized — they present an opinionated perspective and reflect your brand. Point-and-click tools may be easy to use but suffer limited expressivity and power. With code, there’s no limit to what you can create. (See our [D3](https://observablehq.com/@d3/gallery) and [Plot](https://observablehq.com/@observablehq/plot-gallery) galleries for inspiration.)


Suggested change

Good data apps are highly customized — they present an opinionated perspective and reflect your brand. Point-and-click tools may be easy to use but suffer limited expressivity and power. With code, there’s no limit to what you can create. (See our [D3](https://observablehq.com/@d3/gallery) and [Plot](https://observablehq.com/@observablehq/plot-gallery) galleries for inspiration.)

Good data apps are highly customized — they present an opinionated perspective and reflect your brand. Point-and-click tools may be easy to use but suffer limited expressivity and power. With code, there’s no limit to what you can create. (See our [D3](https://observablehq.com/@d3/gallery) and [Plot](https://observablehq.com/@observablehq/plot-gallery) galleries for inspiration.)

should we also mention the supported libraries here, as well as “the whole world of JavaScript libraries”?

Fil · 2024-04-07T13:44:34Z

docs/what-is-framework.md

+
+## Polyglot meets the web
+
+Most application frameworks focus on a single language, such as JavaScript, Python, or R. Framework is different. Framework is _polyglot_: it brings multiple languages together. This approach is especially valuable for data apps where data teams have a preferred language(s) for data analysis but want the full power of JavaScript for interactive graphics. Have your cake and eat it too. 🍰


Suggested change

Most application frameworks focus on a single language, such as JavaScript, Python, or R. Framework is different. Framework is _polyglot_: it brings multiple languages together. This approach is especially valuable for data apps where data teams have a preferred language(s) for data analysis but want the full power of JavaScript for interactive graphics. Have your cake and eat it too. 🍰

Most application frameworks focus on a single language, such as JavaScript, Python, or R. Framework is different. Framework is _polyglot_: it brings multiple languages together. This approach is especially valuable for data apps where data teams have a preferred language (or several) for data analysis but want the full power of JavaScript for interactive graphics. Have your cake and eat it too. 🍰

I’ll just use “preferred languages”.

Fil · 2024-04-07T13:47:17Z

docs/what-is-framework.md

+
+With Framework, editing a Python or R data loader immediately updates the browser preview; no reloading required. Framework’s preview server automatically watches for changes and re-runs the data loader, pushing updates over a socket. And thanks to reactivity, the browser can efficiently incrementally update the display.
+
+Whether your team prefers Python, R, SQL — or even some new language you invented — Framework can give you a best-in-class developer experience and help you build a better data app.


This seems a bit redundant, but could be modified to outline that any piece of software that runs on your computer can be used to build a data app. (I like to use ffmpeg or tesseract as an example, since they're obviously not a "language", but one can immediately imagine how you'd use it to make new datasets.)

Fil · 2024-04-07T14:48:40Z

docs/loaders.md


-Why generate data at build time? Conventional dashboards are often slow or even unreliable because database queries are executed for each viewer on load. By preparing static data snapshots ahead of time during build, dashboards load instantly with no external dependency on your database. You can also optimize data snapshots for what your dashboard needs, further improving performance and offering more control over what information is shared with viewers.
+Why static snapshots? Performance is critical for dashboards: users don’t like to wait, and dashboards only create value if users look at them. Data loaders practically force your app to be fast because data is precomputed and thus can be served instantly — you don’t need to run queries separately for each user on load. Furthermore, data can be highly optimized (and aggregated and anonymized), minimizing what you send to the client. And since data loaders run only during build, your users don’t need direct access to your data warehouse, making your dashboards more secure and robust.


Suggested change

Why static snapshots? Performance is critical for dashboards: users don’t like to wait, and dashboards only create value if users look at them. Data loaders practically force your app to be fast because data is precomputed and thus can be served instantly — you don’t need to run queries separately for each user on load. Furthermore, data can be highly optimized (and aggregated and anonymized), minimizing what you send to the client. And since data loaders run only during build, your users don’t need direct access to your data warehouse, making your dashboards more secure and robust.

Why static snapshots? Performance is critical for dashboards: users don’t like to wait, and dashboards only create value if users look at them. Data loaders practically force your app to be fast because data is precomputed and thus can be served instantly — you don’t need to run queries separately for each user on load. Furthermore, data can be highly optimized (and aggregated and anonymized), minimizing what you send to the client over the network. And since data loaders run only during build, your users don’t need direct access to your data warehouse, making your dashboards more secure and robust.

I’m not sure what “over the network” adds here. My point isn’t so much minimizing network traffic as it is information sharing — not revealing more information than necessary to produce the visualization/intended analysis.

Fil · 2024-04-07T14:50:29Z

docs/loaders.md


 A data loader can be as simple as a shell script that invokes [curl](https://curl.se/) to fetch recent earthquakes from the [USGS](https://earthquake.usgs.gov/earthquakes/feed/v1.0/geojson.php):

 ```sh
 curl https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson
 ```

-Observable Framework uses [file-based routing](./routing), so assuming this shell script is named `quakes.json.sh`, a `quakes.json` file is then generated at build time. You can access this file from the client using [`FileAttachment`](./javascript/files):
+Data loaders use [file-based routing](#routing), so assuming this shell script is named `quakes.json.sh`, a `quakes.json` file is then generated at build time. You can access this file from the client using [`FileAttachment`](./files):


Suggested change

Data loaders use [file-based routing](#routing), so assuming this shell script is named `quakes.json.sh`, a `quakes.json` file is then generated at build time. You can access this file from the client using [`FileAttachment`](./files):

Data loaders use [file-based routing](#routing), so assuming this shell script is named `quakes.json.sh`, a `quakes.json` file is then generated at build time. You can access this file from the client using [`FileAttachment`](./files):

```js echo

FileAttachment("quakes.json")

```

and access its contents with the appropriate method:

Not sure this is necessary here because it repeats the example from the linked FileAttachment docs, and it’s fairly rare to reference FileAttachment without loading the contents, and then it also feels like it needs the explanation that calling FileAttachment by itself doesn’t load the file.

docs/files.md

Fil · 2024-04-07T14:59:18Z

docs/sql.md

@@ -5,7 +5,9 @@ sql:

 # SQL <a href="https://github.com/observablehq/framework/releases/tag/v1.2.0" class="observablehq-version-badge" data-version="^1.2.0" title="Added in v1.2.0"></a>

-Observable Framework includes built-in support for client-side SQL powered by [DuckDB](./lib/duckdb). You can use SQL to query data from [CSV](./lib/csv), [TSV](./lib/csv), [JSON](./javascript/files#json), [Apache Arrow](./lib/arrow), [Apache Parquet](./lib/arrow#apache-parquet), and DuckDB database files, which can either be static or generated by [data loaders](./loaders).
+<div class="tip">This page covers client-side SQL using DuckDB. To run a SQL query on a remote database such as PostgreSQL or Snowflake, use a <a href="./loaders">data loader</a>.</div>


Fil · 2024-04-07T15:31:39Z

docs/deploying.md

@@ -64,6 +64,8 @@ jobs:
          OBSERVABLE_TOKEN: ${{ secrets.OBSERVABLE_TOKEN }}
 ```

+<div class="tip">As shown above, deploy messages can be set using <code>--message</code>. This is especially useful for continuous deployment from a git repository: the message can include the SHA, author, and message of the latest commit.</div>


it makes much more sense here!

mbostock · 2024-04-07T16:44:50Z

Thanks for all the feedback! I’m headed down to Monterey to visit the aquarium but I’ll take a deeper look tonight.

Co-authored-by: Philippe Rivière <[email protected]>

CAYdenberg

I think this is much better. It would still be nice to see some kind of reference section for:
CLI
The standard library/built-ins
but maybe that can all be searched easily enough

CAYdenberg · 2024-04-11T18:08:34Z

docs/what-is-framework.md

+
+Observable Framework — or “Framework” for short — is an open-source static-site generator for data apps. By *data app* we mean an application that is primarily a display of data. Data apps help you derive insights (to understand) and evaluate potential decisions (to take action).
+
+A data app might be a set of coordinated **interactive visualizations** for “self-service” analysis, perhaps to explore a computational model or to investigate activity;


The semicolons at the end of paragraphs feel off; consider just bracket with ellipses

CAYdenberg · 2024-04-11T18:09:57Z

docs/what-is-framework.md

+
+Good data apps are highly customized — they present an opinionated perspective and reflect your brand. Point-and-click tools may be easy to use but suffer limited expressivity and power. With code, there’s no limit to what you can create. (See our [D3](https://observablehq.com/@d3/gallery) and [Plot](https://observablehq.com/@observablehq/plot-gallery) galleries for inspiration.)
+
+Modern development is a marvel. Framework is free and open-source, and projects are just local files, making it easy to incorporate into your existing workflow. Use your preferred editor, source control, and code review system. Write unit tests. Run linters. Automate builds with continuous integration or deployment. Work offline. Self-host. Generate or revise content programmatically with AI. You can do it all.


First sentence doesn't seem on-topic - not really specific to Framework.

CAYdenberg · 2024-04-11T18:14:17Z

docs/getting-started.md

@@ -98,15 +97,15 @@ If you prefer Yarn, run:

 You can run the above command anywhere, but you may want to `cd` to your `~/Development` directory first (or wherever you do local development).

-The first prompt asks where to create your new project. Enter `hello-framework` to create a directory named `hello-framework` within the current directory. Or just hit Enter, as this is conveniently the default. (You can create a project in a different directory by entering a relative or absolute path; on macOS or Linux, such paths start with `../` or `~/` or `/`.)
+The first prompt asks where to create your new project. Enter `hello-framework` to create a directory named `hello-framework` within the current directory. Or just hit Enter, as this is the default. (You can create a project in a different directory by entering a relative path; on macOS or Linux, such paths start with `../` or `~/` or `/` _etc._)


Honestly, listing out all these prompts still feels very tedious to me, given it's literally walking the user through. But maybe for folks less familiar with getting JS projects up and running it might be necessary.

CAYdenberg · 2024-04-11T18:16:53Z

docs/getting-started.md

@@ -338,7 +333,7 @@ const [longitude, latitude] = location;

 To personalize this code snippet to your current location, edit the <code>longitude</code> and <code>latitude</code> values above, or click the **Locate me** button above.

-<div class="caution">NWS does not provide forecasts for points outside the United States, so if you specify such a location the API will return an error and the data loader will fail.</div>
+<div class="caution">NWS does not provide forecasts for points outside the United States. If you specify such a location the API will error and the data loader will fail.</div>


I wonder if a different example would catch a wider set of users? (My Canada is showing ...)

CAYdenberg · 2024-04-11T18:23:01Z

docs/getting-started.md

@@ -487,7 +484,7 @@ Now you can call `temperaturePlot` to display the forecast anywhere on the page:
 display(temperaturePlot(forecast));
 ```

-<div class="tip">JavaScript can be extracted into standalone modules (<code>.js</code> files) that you can <a href="./javascript/imports">import</a> into Markdown. This lets you share code across pages, write unit tests for components, and more.</div>
+<div class="tip">JavaScript can be extracted into standalone modules (<code>.js</code> files) that you can <a href="./imports">import</a> into Markdown. This lets you share code across pages, write unit tests for components, and more.</div>


Suggest removing the "and more" clause. It doesn't really add anything.

CAYdenberg · 2024-04-11T18:37:38Z

docs/javascript.md


 ```js
 1 + 2
 ```

-Note that JavaScript fenced code blocks do not echo their code by default. If you want to show the code, use the `echo` directive:
+A program block looks like this (note the semicolon):


... and the use of const?

CAYdenberg · 2024-04-11T18:42:48Z

docs/javascript.md

 ```

-A program block looks like this:
+Code blocks automatically re-run when referenced [reactive variables](./reactivity) change, or when you edit the page during preview. The block below references the built-in variable `now` representing the current time in milliseconds; because `now` is reactive, this block runs sixty times a second and each each new span it returns replaces the one previously displayed.


This example requires readers to understand inline style attrs, hsl, and javascript Dates. Seems a bit heavy for folks coming from non-web worlds.

CAYdenberg · 2024-04-11T18:45:09Z

docs/javascript.md

-Unlike code blocks, expressions cannot declare top-level reactive variables.
+Expressions cannot declare top-level reactive variables. To declare a variable, use a code block instead. You can declare a variable in a code block (without displaying it) and then display it somewhere else using an inline expression.
+
+## Explicit display


Inappropriate for minors?

CAYdenberg · 2024-04-11T20:06:17Z

docs/reactivity.md

+});
+```
+
+```js echo


This is example is not working correctly for me (2024/04/11, on Firefox).

This works for me on Firefox, but it depends on an external API (because the point is to demonstrate real-time streaming data over a websocket), so it’s possible either the API was down, or that it’s not accessible on your network e.g. due to firewall restrictions.

CAYdenberg · 2024-04-11T20:14:18Z

docs/theme/generate-themes.ts

@@ -134,7 +134,96 @@ In addition to themes and theme modifiers, there are special aliases:
 - \`light\` - an alias for \`${light}\`
 - \`dark\` - an alias for \`${dark}\`

-On its own, \`default\` is equivalent to \`[light, dark]\` (or \`[${light}, ${dark}]\`). The \`default\` theme is applied by default if you don’t specify any color theme. You can also use \`default\` to combine a specific light or dark theme with the default theme of the opposing mode; for example \`[cotton, default]\` is equivalent to \`[cotton, dark]\`, and \`[coffee, default]\` is equivalent to \`[coffee, light]\`.`;
+On its own, \`default\` is equivalent to \`[light, dark]\` (or \`[${light}, ${dark}]\`). The \`default\` theme is applied by default if you don’t specify any color theme. You can also use \`default\` to combine a specific light or dark theme with the default theme of the opposing mode; for example \`[cotton, default]\` is equivalent to \`[cotton, dark]\`, and \`[coffee, default]\` is equivalent to \`[coffee, light]\`.


I think this page should probably at least mention how to create your own theme.

It’s mentioned right at the top; the answer is to create a custom stylesheet.

mbostock added 23 commits April 1, 2024 07:38

checkpoint docs

58a542b

Merge branch 'main' into mbostock/more-docs

d4b15e7

restructure

98b980f

checkpoint restructure

6c5dcde

checkpoint

49708a6

what is framework; note background

280ad71

Merge branch 'main' into mbostock/more-docs

0f6486d

note margin-trim

71e96fd

more edits

2110902

more edits

5e64b3c

more edits

2b19e98

more edits

9ca5a66

Merge branch 'main' into mbostock/more-docs

5a28435

projects page

0ff621e

more overview

bf3e937

more overview

850a1e4

more edits

05f9163

more edits

170ee4a

more edits

e76628c

more edits

9c071e7

more edits

15bfdc0

more edits

e80a088

more edits

b833f32

mbostock added 6 commits April 5, 2024 08:10

Merge branch 'main' into mbostock/more-docs

8ac3c73

more edits

1c8d8b5

more edits

9683a36

more edits

8873b49

more edits

8e455a2

back to files; more edits

c38aebe

mbostock added 5 commits April 5, 2024 20:45

back to files; more edits

e2e86be

more edits

e1d9e87

more edits

af673e1

more edits

4853e80

more edits

101ae73

mbostock marked this pull request as ready for review April 6, 2024 16:07

self-host sample data

45e046f

mbostock requested a review from Fil April 6, 2024 18:11

mbostock added 5 commits April 6, 2024 16:51

project-structure

01b1aa8

reoder markdown

d6c2c61

fix links

0d6e3f8

fix redirect

2ecf918

fix redirects

7b2e5ca

Fil approved these changes Apr 7, 2024

View reviewed changes

mbostock mentioned this pull request Apr 7, 2024

non-collapsible sections #1208

Merged

mbostock and others added 4 commits April 7, 2024 14:29

Apply suggestions from code review

7506044

Co-authored-by: Philippe Rivière <[email protected]>

edits

3cc8720

more edits

4c37d3f

Update docs/files.md

fde0df6

Co-authored-by: Philippe Rivière <[email protected]>

mbostock enabled auto-merge (squash) April 8, 2024 00:14

mbostock merged commit d774917 into main Apr 8, 2024
4 checks passed

mbostock deleted the mbostock/more-docs branch April 8, 2024 00:16

CAYdenberg reviewed Apr 11, 2024

View reviewed changes

mbostock mentioned this pull request Apr 16, 2024

shorter getting started #1245

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

restructure & expand docs #1187

restructure & expand docs #1187

mbostock commented Apr 2, 2024 •

edited

Loading

mbostock commented Apr 5, 2024

Fil left a comment

Fil Apr 7, 2024

Fil Apr 7, 2024

mbostock Apr 7, 2024

Fil Apr 7, 2024

Fil Apr 7, 2024

mbostock Apr 7, 2024

Fil Apr 7, 2024

mbostock Apr 7, 2024

Fil Apr 7, 2024

Fil Apr 7, 2024

mbostock commented Apr 7, 2024

CAYdenberg left a comment

CAYdenberg Apr 11, 2024

CAYdenberg Apr 11, 2024

CAYdenberg Apr 11, 2024

CAYdenberg Apr 11, 2024

CAYdenberg Apr 11, 2024

CAYdenberg Apr 11, 2024

CAYdenberg Apr 11, 2024

CAYdenberg Apr 11, 2024

CAYdenberg Apr 11, 2024

mbostock Apr 16, 2024

CAYdenberg Apr 11, 2024

mbostock Apr 16, 2024


		## The power of code

		Good data apps are highly customized — they present an opinionated perspective and reflect your brand. Point-and-click tools may be easy to use but suffer limited expressivity and power. With code, there’s no limit to what you can create. (See our [D3](https://observablehq.com/@d3/gallery) and [Plot](https://observablehq.com/@observablehq/plot-gallery) galleries for inspiration.)


		## Polyglot meets the web

		Most application frameworks focus on a single language, such as JavaScript, Python, or R. Framework is different. Framework is _polyglot_: it brings multiple languages together. This approach is especially valuable for data apps where data teams have a preferred language(s) for data analysis but want the full power of JavaScript for interactive graphics. Have your cake and eat it too. 🍰


		With Framework, editing a Python or R data loader immediately updates the browser preview; no reloading required. Framework’s preview server automatically watches for changes and re-runs the data loader, pushing updates over a socket. And thanks to reactivity, the browser can efficiently incrementally update the display.

		Whether your team prefers Python, R, SQL — or even some new language you invented — Framework can give you a best-in-class developer experience and help you build a better data app.


		Why generate data at build time? Conventional dashboards are often slow or even unreliable because database queries are executed for each viewer on load. By preparing static data snapshots ahead of time during build, dashboards load instantly with no external dependency on your database. You can also optimize data snapshots for what your dashboard needs, further improving performance and offering more control over what information is shared with viewers.
		Why static snapshots? Performance is critical for dashboards: users don’t like to wait, and dashboards only create value if users look at them. Data loaders practically force your app to be fast because data is precomputed and thus can be served instantly — you don’t need to run queries separately for each user on load. Furthermore, data can be highly optimized (and aggregated and anonymized), minimizing what you send to the client. And since data loaders run only during build, your users don’t need direct access to your data warehouse, making your dashboards more secure and robust.


		Observable Framework — or “Framework” for short — is an open-source static-site generator for data apps. By data app we mean an application that is primarily a display of data. Data apps help you derive insights (to understand) and evaluate potential decisions (to take action).

		A data app might be a set of coordinated interactive visualizations for “self-service” analysis, perhaps to explore a computational model or to investigate activity;


		Good data apps are highly customized — they present an opinionated perspective and reflect your brand. Point-and-click tools may be easy to use but suffer limited expressivity and power. With code, there’s no limit to what you can create. (See our [D3](https://observablehq.com/@d3/gallery) and [Plot](https://observablehq.com/@observablehq/plot-gallery) galleries for inspiration.)

		Modern development is a marvel. Framework is free and open-source, and projects are just local files, making it easy to incorporate into your existing workflow. Use your preferred editor, source control, and code review system. Write unit tests. Run linters. Automate builds with continuous integration or deployment. Work offline. Self-host. Generate or revise content programmatically with AI. You can do it all.

restructure & expand docs #1187

restructure & expand docs #1187

Conversation

mbostock commented Apr 2, 2024 • edited Loading

mbostock commented Apr 5, 2024

Fil left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbostock commented Apr 7, 2024

CAYdenberg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbostock commented Apr 2, 2024 •

edited

Loading