Skip to content

markdown: Add support for ![alt text](url) syntax for images. #29300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

N-Shar-ma
Copy link
Collaborator

@N-Shar-ma N-Shar-ma commented Mar 14, 2024

markdown: Render inline images as small lightbox previews in place.

Images added using the ![alt text](url) syntax are rendered in a fixed size, expandable via lightbox, exactly like the linked image previews, except inline.

For drafts which are only rendered by the frontend markdown processor, images are rendered as a regular link, for now.

markdown: Add support for ![alt text](url) syntax for inline images.

The standard markdown syntax for images is now supported in Zulip. Unlike linked image previews (which is the only way images could be added so far), images added using the ![alt text](url) syntax are truly inline, so are placed in the message exactly where the syntax is, not in a separate paragraph.

Fixes: #28912.

upload: Rename variable filename_url to syntax_to_insert.

This variable holds the markdown syntax for a file that's to be inserted into a message, hence the new name.

copy_and_paste: Fix bug where an unlabelled image could be pasted twice.

This commit removes the unnecessary condition for a linked image to have a label when checking whether to skip an image preview if its generating link has already been copied.

Screenshots and screen captures:
Composing:
image
Previewing:
image
Drafted:
image

Self-review checklist
  • Self-reviewed the changes for clarity and maintainability
    (variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

  • Explains differences from previous plans (e.g., issue description).
  • Highlights technical choices and bugs encountered.
  • Calls out remaining decisions and concerns.
  • Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

  • Each commit is a coherent idea.
  • Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

  • Visual appearance of the changes.
  • Responsiveness and internationalization.
  • Strings and tooltips.
  • End-to-end functionality of buttons, interactions and flows.
  • Corner cases, error conditions, and easily imagined bugs.

@timabbott
Copy link
Member

When writing tests https://spec.commonmark.org/0.18/#images, may be a good source of things to try. Since we're using upstream code, we may not find it easy to match CommonMark on everything, but it should be cheap to at least know where we're diverging.

Comment on lines 2160 to 2162
if not any(src.endswith(ext) for ext in IMAGE_EXTENSIONS):
# If the url is not a previewable image, gracefully fall back to a regular link
return self.link_processor.handleMatch(m, data)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ![…](…) syntax indicates that the user knows this is an image. Not all images use an extension. We should give the user the image they asked for, not pretend we know “better”.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd observed the convert-unsupported-image-to-link behavior here in GitHub actually, but I now understand the merits of letting it be as is, thank you!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give an example from GitHub of this detail?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@timabbott On writing something like ![image](image.png) here in github, it renders as a plain link. Example: image

@N-Shar-ma
Copy link
Collaborator Author

I've addressed most of the feedback above, most importantly moved the html wrangling for presentation of inline images to the frontend, but I'm still working on refining things further.

I'm yet to add backend tests, write a proper commit message, potentially split this across multiple commits, etc

@N-Shar-ma N-Shar-ma force-pushed the image-md branch 2 times, most recently from 05aa2f9 to ec6644f Compare March 20, 2024 08:07
@N-Shar-ma
Copy link
Collaborator Author

Have added tests and cleaned up the code and commit structure, this is ready for another review!

@N-Shar-ma
Copy link
Collaborator Author

When writing tests https://spec.commonmark.org/0.18/#images, may be a good source of things to try. Since we're using upstream code, we may not find it easy to match CommonMark on everything, but it should be cheap to at least know where we're diverging.

The reference style multiline syntax does not work, and any text inside [] is taken as is, so any inline syntax inside is copied to the image's alt attribute verbatim. These 2 are the main areas the implementation diverges from the spec. Other things like specifying the title attribute works as expected (see the added tests)

@N-Shar-ma N-Shar-ma force-pushed the image-md branch 2 times, most recently from 55a133c to d226bbe Compare April 2, 2024 17:04
@N-Shar-ma
Copy link
Collaborator Author

@zulipbot add "chat.zulip.org. review"

@zulipbot
Copy link
Member

zulipbot commented Apr 5, 2024

ERROR: Label "chat.zulip.org. review" does not exist and was thus not added to this pull request.

@N-Shar-ma
Copy link
Collaborator Author

@zulipbot add "chat.zulip.org review"

@zulipbot zulipbot added the chat.zulip.org review Added by maintainers when a PR requires chat.zulip.org testing to proceed. label Apr 5, 2024
@timabbott
Copy link
Member

I merged the first two commits as the series ending with 633d814.

@timabbott
Copy link
Member

So if you upload 3 images at once via dragging them or via selecting multiple files in the image picker, one gets two regression-looking outcomes:

  • The images don't get put on the same line with each other, resulting in them stacking vertically rather than horizontally.
  • The message controls seem to get displaced downwards in a message only containing such images.

image

@N-Shar-ma
Copy link
Collaborator Author

The images don't get put on the same line with each other, resulting in them stacking vertically rather than horizontally.

That is because we always put every upload (whether image or any other allowed format) in it's own line, so that's how they show up. What exact alternative behaviour are you proposing here @timabbott?

@alya
Copy link
Contributor

alya commented Apr 13, 2024

Hm, yeah, the image arrangement might be OK?

@N-Shar-ma
Copy link
Collaborator Author

The message controls seem to get displaced downwards in a message only containing such images.

I'm not sure why exactly it is needed currently, but removing the baseline alignment for .messagebox-content introduced in this commit seems to fix this. @karlstolley would appreciate your insight here as the author of this commit!

@karlstolley
Copy link
Contributor

That commit is necessary for the overall grid alignment in the message box. What I think needs to be applied here are the styles introduced in #26942 to the &:has(> .message_inline_image:first-child) selector to use the new paragraph-nested forms introduced here:

    &:has(> .message_inline_image:first-child),
    &:has(> p > .message_inline_image:first-child) {
        align-self: center;
    }

It might be possible to tidy up and simplify that selector, but we'd have to watch very carefully for similar regressions that that might introduce.

Also, it does appear in testing that the fallback work with @supports already addresses this case, so no further adjustment is needed there.

@timabbott
Copy link
Member

For image arrangement, I think if you drag multiple images at once or select multiple in your browser in a single operation, it's probably more correct to put the images on a line together. Not sure if that's easy, but it was a major UX improvement when we made multiple linked images wrap on a single line (#20975), and this PR effectively undoes that feature from a practical standpoint.

@N-Shar-ma
Copy link
Collaborator Author

@karlstolley Thanks a lot for linking to that PR, I faintly remembered it but couldn't find it.

Since the problem was not for inline images that were 1st in their paragraph, but for any 1st paragraph containing an inline image, I've added the selector &:has(> p:first-child > .message_inline_image). I tested extensively manually and it seems to work fine with a variety of message compositions.:

image

@karlstolley
Copy link
Contributor

karlstolley commented Apr 15, 2024

If there's any text at all, that could give you a false negative. Test with just the image by itself, without any text. Your selector looks good to me, but I just want to make sure you're testing under the right/problem circumstances.

@N-Shar-ma
Copy link
Collaborator Author

I did test for that too, just didn't include it in the screenshot.

image

@N-Shar-ma
Copy link
Collaborator Author

N-Shar-ma commented Apr 15, 2024

@timabbott I've added a prep commit that puts all uploads made in one go on the same line. This means for all kinds of uploads, not just images, each link text will be inserted inline together all in 1 block, while the previews will be as before (horizontally stacked in another paragraph). For the new ![alt text](url) images, this means they too will stack horizontally when uploaded together.

Is this okay?

@timabbott
Copy link
Member

Hmm, this might be fairly ugly in the case that you upload a mixture of previewable elements and non-previewable elements.

@N-Shar-ma
Copy link
Collaborator Author

@timabbott That's true ☹️ ... How about we only put them on 1 line if all the uploads are images (ie, anything we use the ![]() syntax for)?

@timabbott
Copy link
Member

Yeah, that rule is probably a solid one.

@N-Shar-ma
Copy link
Collaborator Author

@timabbott Made the change! Now only for uploads with multiple image-only type files, are the files all added in their own line, so they render as being stacked horizontally

@N-Shar-ma N-Shar-ma requested a review from timabbott April 27, 2024 21:47
@timabbott timabbott added the deployed on chat.zulip.org Added by maintainers when a PR is currently being tested on chat.zulip.org. label May 2, 2024
@timabbott timabbott removed the deployed on chat.zulip.org Added by maintainers when a PR is currently being tested on chat.zulip.org. label May 14, 2024
N-Shar-ma added 3 commits June 6, 2024 02:23
The standard markdown syntax for images is now supported in Zulip.
Unlike linked image previews (which is the only way images could be
added so far), images added using the `![alt text](url)` syntax are
truly inline, so are placed in the message exactly where the syntax is,
not in a separate paragraph.

Fixes: zulip#28912.
So far, each file was always inserted on its own line. Now for images
uploaded together, they're inserted inline, all in a single block.

This is so that images stack horizontally when uploaded together.
Images added using the `![alt text](url)` syntax are rendered in a fixed
size, expandable via lightbox, exactly like the linked image previews,
except inline.

For drafts which are only rendered by the frontend markdown processor,
images are rendered as a regular link, for now.
@zulipbot
Copy link
Member

Heads up @N-Shar-ma, we just merged some commits that conflict with the changes you made in this pull request! You can review this repository's recent commits to see where the conflicts occur. Please rebase your feature branch against the upstream/main branch and resolve your pull request's merge conflicts accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chat.zulip.org review Added by maintainers when a PR requires chat.zulip.org testing to proceed. has conflicts size: XL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement CommonMark image syntax
6 participants