Yet another rss reader

For Reviewers

Building Raton has been tested using MacOS Ventura 13.2.1 on an Apple Silicon M1 chip (and on MacOS Big Sur on an intel chip) and requires Node.js 18.15.0, yarn version 1.22.17, any recent version of bash or zsh (version GNU bash, version 3.2.57(1)-release (arm64-apple-darwin22) is shipped with Ventura and used) and jq version jq-1.6.

To install the dependencies: yarn install To build the extension into dist/: ./bin/release

Feed Article Sanitization

In ./src/components/article_browser/ArticleBrowser.svelte you might see the line:

{@html $selectedArticle.content}

Which is Sveltes way of basically doing element.innerHTML = stuff; While normally potentially dangerous, this content is pre-sanitized during the fetching and processing of feeds, in ./src/lib/feeds/transforms/sanitizer.ts using the DOMPurify package.

Development

Raton is built with mainly: Typescript and Svelte, TailwindCSS is used for most styling, Prettier is used for formatting, ~Rollup~Vite.js for bundling. Storage of articles is done with Dexie and IndexedDB, parsing is with some hand-rolled parsers in lib/feeds.

Credits:

Setup

An asdf .tool-versions file is provided with the recommended node version. Dependencies are managed through yarn:

yarn install

Developing

Raton is built and bundled with Vite.js as it provides the best experience with Svelte's own tools and produces the best bundles that keep Mozilla's linter happy compared to Parcel v2 and other bundlers.

The dev command will start Vite.js watching files and automatically rebuild the extension on changes:

yarn dev

Additionally, Storybook is setup with the Svelte CSF format for documenting, previewing and rapidly developing components; Check out any of the .stories.svelte files for examples and run Storybook with:

yarn storybook

To kick off a full production grade build of the extension's JS and CSS into dist/, use the build command which will add some treeshaking and remove source maps, etc:

yarn build

Known To Dos

(These should eventually be tracked in fossil better, but for now they're here.)

This list is mostly ordered generally by priority as I see it:

  • [ ] Rework review and building scripts to remove Fossil integrations where needed and make sure Vite.js is used now

  • [ ] Document the feed parsing pipeline

  • [X] Basic Feed Management

    • [X] Add feed
    • [X] Remove feed
    • These are working but the UX is a little wonky still

  • [-] Bulk article management (bulk read/unread at least)

    • Bulk "mark as read" is working

  • [X] OPML Import/Export

    • Working but probably bug filled

  • [X] Explicit refresh of articles and fetching feeds

  • [X] Refresh of article list

    • need to see about keeping the scroll at the same location

  • [X] Keyboard shortcuts

  • [-] Settings panels

    • [X] General Settings
    • Preferred locale?
      • Don't have additional locales yet, so this doesn't make much sense for now
    • [X] Configurable keyboard shortcuts
    • [X] Credits Panel
    • [ ] Tutorial/Help panels
    • [X] Update News panel
    • Something like? Keep a Changelog

  • [-] Documentation on building and reviewing

  • [-] Guess I should write some tests or something

  • [-] Debugging tools

    • A DB exporter/importer exists in the settings page
    • development builds should have additional logging turned on
    • several pages have JSON views of their relevant data at the bottom of the page in development builds

  • [X] Storybook document the various components for easier development/styling work

    • Work in progress as components get written but this is setup and working

  • [X] Article read/unread functionality

  • [X] Feeds in sidebar, view articles on a per feed basis

    • [X] Unread badges/chips

  • [X] Limited loading of chunks of articles within a virtualized list

    • The current library used for this, svelte-tiny-virtual-list has some design issues with the scrollTo setup, so this will most likely be replaced in the future with something that handles a more event driven scrollTo setup.

  • [X] Better build script that walks through the steps to ensure things are correctly setup and ready to go for a new version

Road Map Features

  • [ ] Unsubscribing from feeds should give the option to keep their articles

  • [-] Alternative article list views

    • [ ] Feed view like this one from tailwind ui
    • [X] Tri-column view
    • Several UX bugs with this that need worked out still however

  • [ ] Bug reporting with sentry or similar

    • Need a privacy statement and explanation on what data is captured

  • [ ] Renaming/editing feed titles

  • [ ] Dark mode support

  • [ ] Advanced importing to be able to select which feeds from the import to subscribe to

  • [ ] Feed organization with "folders"

  • [ ] Advanced filtering and sorting of articles (filter by unread, sort by title, sort by date fetched)

  • [ ] Feedback system

    • Like VS Codes: :) or :( emoji and a free form text input?
    • Where to send feedback?
    • Could just redirect to the forum in fossil or something
    • Sentry also has a provide feedback form setup

Known Bugs

  • [ ] Feed and article views will jump around when the background page issues a refresh notification
  • [ ] Does not work in Chrome because the background page needs access to the DOM Api's for pasing the feeds
    • Fuck you too, MV3 service workers
  • [ ] Some feeds don't have titles so things get a little blank looking
  • [ ] Subscribing/Unsubscribing from feeds is still pretty weird, slow or otherwise odd

Ideas

These need to be fleshed out a bit before they're ready to be ditched or moved to the roadmap as new features.

  • Custom Branding

  • Gumroad membership/subscription fee to use? just like $1/m or $2/m

  • Custom fetch interval?

    • Over-ridable per feed

  • Command palette style interface for switching feeds quickly

  • Tooling to convert a markdown changelog to something that's usable in the release notes section

  • Tooling to automatically make the credits page for libraries used?

  • Details on the feed management page for geeks and nerds

    • Total items stored
    • subscribed on
    • feed type
    • image/additional channel details
    • last checked on

  • Better feedback for (un)subscribing to feeds?

  • Article & Feed search

  • Staring articles (to avoid deleting and being able to find them later)

  • Deleting/hiding articles

  • Specialized handling of lobst.rs and hackernews feeds to have tabs for "Article in an iframe" and "comments in an iframe"

    • Maybe some sort of plugin setup for custom handling of different feeds
    • This is somewhat done where comment links show up in the article viewer
    • Specialized handling of feeds would also be nice to apply custom styling or better media extraction for feeds like APOD

  • YouTube feed handling to load the video

  • Podcast feed handling to load the audio

  • Notifications for new items from all or specific feeds

  • Pruning of feed articles after X time

  • Feed Discovery Page action

    • Could just open the app to /manage/add?url= for starters

  • "Open in Raton" option, or some indicator for viewing websites that are an article in the DB

  • Configurable color schemes

Testing

Tests make use of AVA as the runner and assertion library. ESBuild is used as the loader to allow tests to be written in TypeScript still. The Integration and System tests use some custom type adustments and several packages for spinning up the environment to mock the web-extension API or to start the browser and load the extension. See (read: self-discover because I haven't documented any of it) those directories for additional help and tooling.

Unit

yarn test:unit

These are the fastest and should be covering more of the business logic. General rule of thumb is if business logic is living in a component file that makes it hard to unit test: move it out into it's own function/module.

Resources:

Integration

yarn test:integration

Spins up JSDom environments for each context to test general integration between them without needing the full browser. These environments also include mocking of the browser api and some other helpful tooling. Check out tests/integration/setup.ts for more information.

Resources:

System

yarn test:system

Spins up a full browser instance to test the extension extensions behavior; These are extremely slow, however, and should be limited to E2E unless logical. Check out tests/system/setup.ts for more information.

A couple of helpers are provided as well:

async switchToBrowserChrome(geckodriver) - Switchs the selenium web driver to control the browser chrome async switchToBrowserContent(geckodriver) - Switchs the selenium web driver to control the current window/tab's content. Needed for navigations and for interacting with the content script UI async navigateTo(geckodriver, path) - Navigates the browser and sets the selenium web driver to controlling the browser chrome. Handles relative paths from tests/system/pages/ to help test the content scripts against various page set ups. async takeScreenshot(geckodriver) - Takes a screenshot of the window and saves it to ./screenshot-<date>.png

Resources:

Production Builds

Release Walkthrough Script

This script walks you through the process of cutting a release to upload to the stores, and helps automate some of the steps too.

./bin/release-walkthrough

Manually

Pretty simple, first ensure that you've updated the release notes:

  • Release notes are currently in ./src/components/settings/release_notes as a svelte component per version
  • Make sure it's pulled in and rendering in ./src/components/settings/ReleaseNotes.svelte
  • Increment the version in package.json to the correct new version matching the release notes

When all done, run:

./bin/release

This will build the extension in production mode, and produce the AMO ready zip in web-ext-artifacts/raton-<version>.zip.

Fossil 2.17 [f48180f2ff] 2021-10-09 14:43:10
This page was generated in about 0.005s