fixup: address review comments

Author: gireeshpunathil

documentation/crash/crash_setup.md

@@ -1,5 +1,5 @@
 
-## Node.js application crash diagnostics: Best Practices series #1
+# Node.js application crash diagnostics: Best Practices series #1
 
 This is the first of a series of best practices and useful tips if you
 are using Node.js in large scale production systems. 
@@ -23,7 +23,31 @@ need to be manually `prepared` in advance to enable crash dianostic
 data generation on the first failure itself, without loosing vital data.
 The rest of the document illustrates this preparation steps.
 
-## Available  disk space
+The key artifacts for exploring Node.js application crashes are:
+ - core dump (a.k.a system dump, core file)
+ - diagnostic report (originally known as node report)
+
+Reference: [Diagnostic Report](https://nodejs.org/dist/latest-v12.x/docs/api/report.html)
+
+## Common issues
+
+While the said key artifacts are expected to be generated on abnormal
+program conditions such as crash, (diagnostic report is still
+experimental so requires explicit command line flags to switch it ON)
+there are a number of issues that affects the automatic and complete
+generation of these artifacts. Most common such issues are:
+ - Insufficient disk space for writing core dump data
+ - Insufficient privilege to the core dump generator function
+ - Insufficient resource limits set on the user
+ - In case of diagnostic report, absence of report and symtpom flag
+
+## Recommended Best Practice
+
+This section provides specific recommendations for
+how to configure your systems in advance in order to be
+ready to investigate crashes.
+
+### Available  disk space
 Ensure that there is enough disk space available for the core file
 to be written:
 
@@ -44,31 +68,38 @@ $ top -p 106916
 106916 user       20   0  600404  54500  15572 R 109.7  0.0  81098:54 node    
 ```
 
-In Darwin, the flag is `-pid`
-In AIX, the command is `topas`
+In Darwin, the flag is `-pid`.
+
+In AIX, the command is `topas`.
+
 In freebsd, the command is `top`. In both AIX and freebsd, there is no
-flag to show per-process details. In Windows, you could use the task
+flag to show per-process details.
+
+In Windows, you could use the task
 manager window and view the process attributes visually.
 
 Insufficient file system space will result in truncated core files,
 and can severely hamper the ability to diagnose the problem.
 
 Figure out how much free space is available in the file system:
+
 `df -k` can be used invariably across UNIX platforms.
+
 In Windows, Windows explorer when pointed to a disk partition,
 provides a view of the available space in that partition.
 
-## Core file location and name
-
 By default, core file is generated on a crash event, and is
 written to the current working directory - the location from
 where the node process was started, in most of the UNIX variants.
+
 In Darwin, it appears in /cores location.
 
 By default, core files from node processes on Linux are named as
 `core` or `core.<pid>`, where <pid> is node process id.
+
 By default, core files from node processes on AIX and Darwin are
 named ‘core’.
+
 By default, core files from node processes on freebsd are named
 ‘%N.core’. where `%N` is the name of the crashed process.
 
@@ -80,7 +111,20 @@ Modify pattern using `sysctl -w kernel.core_pattern=pattern` as root.
 
 In AIX, `lscore` shows the current core file pattern.
 
+A best practice is to remove old core files on regular intervals.
+
+This makes sure that the space in the system is used efficiently,
+and no application spefific data is persisted inadvertently.
+
+A best practice is to name core file with the name, process ID and
+the creation timestamp of the failed process.
+
+This makes it easy to relate the binary dump with crash specific context.
+
+### Configuring to ensure core generation
+
 Enable full core dump generation using `chdev -l sys0 -a fullcore=true`
+
 Modify the current pattern using `chcore -p on -n on -l /path/to/coredumps`
 
 In Darwin and freebsd, `sysctl kern.corefile` shows the corrent core file pattern.
@@ -90,7 +134,9 @@ Modify the current pattern using `sysctl -w kern.corefile=newpattern` as root.
 To obtain full core files, set the following ulimit options, across UNIX variants:
 
 `ulimit -c unlimited` - turn on core file generation capability with unlimited size
+
 `ulimit -d unlimited` - set the user data limit to unlimited
+
 `ulimit -f unlimited` - set the file limit to unlimited
 
 The current ulimit settings can be displayed using:
@@ -104,15 +150,37 @@ system administrator. System administrators (with superuser privileges)
 may display, set or change the hard limits by adding the -H flag to
 the standard set of ulimit commands.
 
-## Manual dump generation
+For example with:
+`ulimit -c -H`
+
+104857600
+
+we cannot increase the core file size to 200 MB. So
+
+`ulimit -c 209715200`
+
+will fail with reason:
+
+`ulimit: core size: cannot modify limit: Invalid argument`
+
+So if you hard limit settings are constraining your application's
+requirement, relax those specific settings through administrator
+account.
+
+## Additional information
+
+### Manual dump generation
 
 Under certain circumstances where you want to collect a core
 manually follow these steps:
 
 In linux, use `gcore [-a] [-o filename] pid` where  `-a`
 specifies to dump everything.
+
 In AIX, use `gencore [pid] [filename]`
+
 In freebsd and Darwin, use `gcore [-s] [executable] pid`
+
 In Windows, you can use `Task manager` window, right click on the
 node process and select `create dump` option.