Pre-release merge for submission v0.1-alpha

[erickpeirson]

Review will close at 1pm on Tuesday, June 4. Subsequent reviews will take place on an ongoing basis during alpha and beta testing.

This is a pre-alpha merge for v0.1 of the submission UI. The original goals for this release were:

• Provide all existing services of classic system (feature-parity), with seamless integration with other classic components (e.g. moderation, announcement).
• Modest improvement to user experience.
• Move submission UI to new unified design paradigm (color scheme, fonts).
• Focus on reducing confusion, improving messaging to address problems related to frequent user support issues.
• Under the hood, implement a new data architecture that better scales and evolves to meet user and stakeholder needs.

The latter objective is covered by arXiv/arxiv-submission-core#64

Some of the major differences from the classic UI include:

• The submission process is broken down into smaller steps
• JREF, withdrawal, and cross-list requests begin to be treated differently than new submissions and replacements
• Highlighting of key terms/phrases in the TeX compilation log; this is the first incremental move toward a more user-friendly compilation experience (e.g. taxonomy of known errors, distilled log output, etc)
• HTML email confirmation
• Native UTF8 support

Under the hood, we have broken out file management (upload), compilation, and plain text extraction as stand-alone services. See the arXiv submission docs for an overview.

Practically speaking, this means that:

• the UI is calling the file management and compilation services behind the scenes, and
• the UI is no longer responsible for orchestrating text extraction and calling the autoclassifier -- this is now handled by the submission agent. So is sending the confirmation e-mail.

How to review

Per the arXiv NG release process, alpha means that:

1. The majority of and/or highest priority functional goals for the release have been met.
2. Bugs are expected to exist, and known bugs may not yet have been fully addressed.
3. The release is deployed for internal review. This may be on the on-premises “beta” server, or in a staging environment in the cloud.
4. Dev and operations teams all participate in review, and report bugs, improvement requests, and other feedback to the feature team. A two-stage alpha review will first target the core operations team, and then broaden testing to volunteer moderators and close partners.
5. Feature team begins to prioritize and fix bugs, performs load testing, and works to evaluate the release against quality and performance goals.

The first round of alpha testing will involve some volunteers outside the core ops team.

In advance of alpha, therefore, please take a look at the following things:

• If you have not yet done so already, please review the architecture for the arXiv-NG submission system.
• Try spinning up the UI and other submission services using docker-compose. See https://github.com/arXiv/arxiv-submission-ui#quick-start for instructions.
• Please help to find any obvious bugs. Try running some announced source packages through, or take a look at the filemanager tests for lots of oddball submissions. The goal is to avoid anything terribly embarrassing when we enter alpha.

Previews

Submission dashboard

http://127.0.0.1:8000/

Verify contact information

http://127.0.0.1:8000/1/verify_user

Confirm authorship

http://127.0.0.1:8000/1/authorship

Select a license

http://127.0.0.1:8000/1/license

Accept submission policies

http://127.0.0.1:8000/1/policy

Primary classification

http://127.0.0.1:8000/1/classification

Cross-list categories

http://127.0.0.1:8000/1/cross_list

File upload

http://127.0.0.1:8000/1/file_upload

Compile PDF from source

http://127.0.0.1:8000/1/file_process

Core metadata

http://127.0.0.1:8000/1/add_metadata

Optional metadata

http://127.0.0.1:8000/1/add_optional_metadata

Final preview

Submission confirmation

Confirmation email (via MailHog)

You can access mailhog at http://127.0.0.1:8025

[mhl10]

Can these be example.org addresses?

[erickpeirson]

They could... the idea with using mimesis is we're generating random (hopefully) fake addresses. Is the worry that these might actually exist?

[mhl10]

I was just being über paranoid on a Monday--not with how mimesis is being used at bootstrap time (totally fine) but how the real-looking examples are being commemorated in the README (and indexed, etc.).

[erickpeirson]

Good to be paranoid. Created #92

[mhl10]

I would suggest handling BadRequest separately and using a milder generic error message, since this isn't really unexpected and we probably don't want users contacting support in these cases.

[erickpeirson]

Most definitely. https://arxiv-org.atlassian.net/browse/ARXIVNG-2196

[mhl10]

Nit: python's lexicographic sorting feels unnatural for a list of files/folders since it's not how most operating systems (AFAIK) sort files/folder names. I would be inclined to apply string.casefold() before sorting.

[erickpeirson]

Created #83

[mhl10]

It would be nice to have at least one user with limited/no endorsements. I notice also that the only table related to endorsements in the test DB is arXiv_endorsement_domains and it's empty.

[erickpeirson]

Created #90