User:KleaBot/mirror/Discourse

From Archiveteam
Jump to navigation Jump to search

Discourse is a forum/Q&A hybrid. It is both an open-source forum software and a forum hosting service.

The Good

Discourse markets itself as a concoction of the good collaborative environment of old-school forums, and the interactivity of modern websites. It's also open-source, you can run a Discourse forum of your own.

The Bad

Discourse also offers hosting for a very hefty price (~$100 per month). As such, it's very possible that anyone who's using their hosting for their forum could chicken out at any minute, meaning their forum is incredibly volatile. See the Known Victims section for further details.

Archiving

A default Discourse instance uses incremental IDs for channels, topics and replies. There will most likely be instances with exemptions, utilising other ID systems.

To get topics as a JSON with cooked HTML text and other verbose variables, you can use "/t/*.json" (e.g. "https://ytpmania.net/t/15378.json"). You can also get JSON for categories ("https://community.cloudflare.com/c/whats-new/57.json") and users ("https://forum.snapcraft.io/u/igor.json").

The raw markdown text of a topic can be saved with the "/raw/" path ("https://devforum.roblox.com/raw/40471").

A JSON or raw text of a topic can be limited to a certain amount of replies; you can add a reply number to the URL to get a singular reply ("https://discourse.lubuntu.me/raw/3528/9").

A URL that only uses numbers will redirect to a URL with the name included (e.g. "https://forum.opencv.org/t/7" will lead to "https://forum.opencv.org/t/welcome-to-opencv-forum/7"). This does not apply to "/raw/" and "/t/*.json" paths.

Discovery

The URLs project is discovering Discourse instances and storing them in the discourse-inbox queue for a later DPoS Discourse archiving project.

A number of Discourse forums are hosted as subdomains of discourse.group and can be discovered with a web search for 'site:discourse.group'[1].

Lists of instances can also be found at https://builtwith.com/?discourse.

ArchiveBot Usage

Discourse works "reasonably well" in ArchiveBot. Currently it is not covered by the forums igset and may be run without an igset. Playback works in the WBM but JS must be disabled.

Many Discourse instances seem to be rate limited so that they will return 429s after a few minutes, even at the default speed. But they do recover quickly and seem to be fine with --concurrency 2 --delay 500, as long as they aren’t served by the same host, which would happen if both forums are hosted at Discourse. In order to avoid that, there shouldn’t be multiple Discourse jobs running on the same pipeline series.

Please link to this page in the job note when running Discourse forums using ArchiveBot, or at least put in the word "Discourse" somehow.

Notable Discourses

At Risk

  • Homebrew
    Went read-only on 1 January 2021, migrating to GitHub Discussions.

Inactive

From Discourse/inactive (edit)

Active Discourses

From Discourse/active (edit)

Known Victims

  • Imgur Community Forums [4]
  • Udacity Forums
    • Disappeared with the acquisition through Accenture[6], but from what the Wayback Machine saved[7], it seemingly was login-walled from the very beginning.
  • CerealLand Forums
    • This was a community around a Minecraft server that seems to have disappeared around 2022.[8]

Lost Discourses

From Discourse/lost (edit)

List of Archived Discourse Forums

From Discourse/archived (edit)

References

Uncategorized List of Found Discourse Forums

From Discourse/uncategorized (edit)

Flux Health | Healthcare in flux]

References