Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Systematically survey message content for unimplemented features. #917

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Systematically survey message content for unimplemented features. #917

wants to merge 5 commits into from

Conversation

PIG208
Copy link
Member

@PIG208 PIG208 commented Aug 27, 2024

Currently a WIP. The code needs a bit of cleanup.

@PIG208 PIG208 changed the title Systematically survey unimplemented message content features. Systematically survey message content for unimplemented features. Aug 27, 2024
@PIG208 PIG208 force-pushed the pr-check-features branch 4 times, most recently from 95688ba to 5728c5f Compare August 29, 2024 06:25
@PIG208 PIG208 mentioned this pull request Aug 29, 2024
Merged
@PIG208 PIG208 force-pushed the pr-check-features branch 2 times, most recently from e9133c3 to 43682cd Compare August 29, 2024 06:54
@PIG208 PIG208 marked this pull request as ready for review August 29, 2024 06:54
@PIG208 PIG208 added the maintainer review PR ready for review by Zulip maintainers label Aug 29, 2024
@PIG208 PIG208 requested a review from chrisbobbe August 29, 2024 06:56
@PIG208
Copy link
Member Author

PIG208 commented Aug 29, 2024

Should be ready for review once the CI passes

@PIG208 PIG208 force-pushed the pr-check-features branch 2 times, most recently from ba7190e to b0d8d5a Compare August 29, 2024 18:38
@PIG208
Copy link
Member Author

PIG208 commented Aug 29, 2024

Pushed some typo fixes.

@PIG208 PIG208 linked an issue Aug 29, 2024 that may be closed by this pull request
Systematically survey message content for unimplemented features #190
Open
@PIG208 PIG208 removed the maintainer review PR ready for review by Zulip maintainers label Aug 29, 2024
@PIG208 PIG208 removed the request for review from chrisbobbe August 29, 2024 19:09
@PIG208
Copy link
Member Author

PIG208 commented Aug 29, 2024

Still needs to support:

Run it on all public messages in realms that publicly list themselves as open communities. For example, have the script sign up a test user in each realm and log in as that test user.

@chrisbobbe
Copy link
Collaborator

chrisbobbe commented Aug 30, 2024 •
edited
Loading

Exciting!!

Would it be easy to also include the number of messages with each feature, in the output of unimplemented_features_test.dart?

@PIG208
Copy link
Member Author

PIG208 commented Aug 30, 2024

Yeah, it should be straightforward.

@PIG208 PIG208 mentioned this pull request Aug 30, 2024
Open
13 tasks
@PIG208 PIG208 force-pushed the pr-check-features branch 2 times, most recently from fbc0745 to 0bc1c06 Compare September 7, 2024 04:18
@PIG208
content [nfc]: Use single and last instead to access child.
b9dd217 
This improves readability, especially in the .last case.

Signed-off-by: Zixuan James Li <[email protected]>
@PIG208
content: Handle code block elements without grandchildren.
6cf84df 
A code block containing no grandchildren (i.e. a
<div class="codehilite"> element containing an empty <pre> block)
for some very dated messages can crash the parser if not handled.

The affect messages in public https://chat.zulip.org history ranged
from message ID 5020 to 29855, all in 2016.

Signed-off-by: Zixuan James Li <[email protected]>
@PIG208
Copy link
Member Author

PIG208 commented Sep 7, 2024 •
edited
Loading

TODO: Add this to tools/check making it easier to run.

@PIG208
deps: Make args and ini direct dev dependencies.
50e9d0a 
We will later use them to write CLI scripts for fetching messages.

Signed-off-by: Zixuan James Li <[email protected]>
@PIG208 PIG208 force-pushed the pr-check-features branch 6 times, most recently from 6b3e698 to 1007086 Compare September 9, 2024 20:20
@PIG208
tools/content: Support systematically surveying unimplemented content…
b321908 
… features.

We added 2 scripts.

- fetch_messages.dart, the script that fetches messages from a given
  Zulip server, that does not depend on Flutter or other involved
  Zulip Flutter packages, so that it can run without Flutter.  It is
  meant to be run first to produce the corpuses needed for surveying
  the unimplemented features.

  The fetched messages are formatted in JSON Lines format, where each
  individual entry is JSON containing the message ID and the rendered
  HTML content.  The script stores output in separate files for
  messages from each server, because message IDs are not unique across
  them.

- unimplemented_features_test.dart, a test that goes over all messages
  collected, parses then with the content parser, and report the
  unimplemented features it discovered.  This is implemented as a test
  mainly because of its dependency on the content parser, which
  depends on the Flutter engine (and `flutter test` conveniently sets
  up a test device).

  The test can be run manually via:
    `flutter test --dart-define=corpusDir=path/to/corpusDir tools/content`

  See comments from the file for more instructions.

Signed-off-by: Zixuan James Li <[email protected]>
@PIG208
Copy link
Member Author

PIG208 commented Sep 9, 2024

In this revision, I moved to using zuliprc to make it easier to run it with tools/check parser without specifying additional arguments. The new suite is not included in tools/check --all. Just entering tools/check parser should give you sufficient guidance on how to use it.

@PIG208 PIG208 added the maintainer review PR ready for review by Zulip maintainers label Sep 9, 2024
@PIG208
Copy link
Member Author

PIG208 commented Sep 9, 2024 •
edited
Loading

For reviewing this, it might be helpful to test tools/check parser and see if you can get it working without needing to look at the commit message. Because it is meant to be used manually in the future. It will be easier to test with an API key from a low-traffic public Zulip server because CZO would have a lot of messages to fetch.

@PIG208
tools/check: Add suite (message content) parser.
be9d79a 
Some main benefits of adding both tools/content scripts to tools/check
are that you don't need to manage your directories for storing the
zuliprc files and corpuses, or specify any command line options for
fetching message contents and running the unimplemented features test.

The script is intended to be run manually, not as a part of the CI,
because it is very slow, and it relies on some out of tree files like
API configs (zuliprc files) and big dumps of chat history.

Fixes: zulip#190

Signed-off-by: Zixuan James Li <[email protected]>
@gnprice gnprice mentioned this pull request Sep 10, 2024
Open
@PIG208
Copy link
Member Author

PIG208 commented Sep 10, 2024

The public message history from CZO can be downloaded from this protected dropbox link. We probably will think of a better way of reusing this in the future.

@PIG208 PIG208 removed the request for review from chrisbobbe September 13, 2024 17:38
@PIG208 PIG208 removed the maintainer review PR ready for review by Zulip maintainers label Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Systematically survey message content for unimplemented features
2 participants
@PIG208 @chrisbobbe