Heroku

From Archiveteam
Revision as of 06:57, 17 July 2021 by Tech234a (talk | contribs) (Additional suggestions)
Jump to navigation Jump to search

Heroku is an online application hosting platform notable for its generous free tier. Heroku has a feature that can allow people to deploy apps to the platform without leaving their web browser.

Free Tier Details

Execution Hours and App Limits

  • 550 execution hours per month shared across up to 5 apps for accounts without credit card verification (this is about 74% to 82% of a given month)
  • 1000 execution hours per month shared across up to 100 apps for accounts with credit card verification (this is more than an entire month)
  • Execution hour limits seem to only be checked about once per day... so if an unverified account has used 549.99 hours when the daily check occurs all apps on the account will actually continue to run for another day before being stopped.
  • Apps that are enabled on accounts that have exhausted their monthly execution hours will automatically be restarted during the first day of the next month.

RAM

  • Free apps are limited to 512MB of RAM, going beyond this limit prints warning messages in the application logs. Reaching 1024MB (200% of the RAM limit) will cause an app to immediately be terminated and restarted.

Web Dynos

Web applications run on free accounts are shut off if they don't receive a request at least once every 30 minutes. Worker dynos are not subject to this limit and are not exposed to the web (unless something like ngrok is used). (For all platform users: web applications can only listen for incoming web traffic on Heroku-specified ports (HTTP/HTTPS/WebSocket). Worker dynos cannot directly listen for incoming traffic on any port.)

Platform Details

Bandwidth

All users are soft-limited 2TB of bandwidth per month[1]. This includes data downloaded and uploaded.

Storage

Apps have essentially unlimited temporary hard-disk storage. Storage is cleared whenever an app is restarted (by the user or platform) or redeployed.

Daily Restarts

Heroku automatically restarts apps every 24-27.6 hours when they are running.

Environment Variables

Environment variables can be set in Heroku app settings. Updating an environment variable restarts the container.

Shutdown Warning

Heroku always sends a warning signal (SIGTERM) 30 seconds before terminating a container, including when completing a user-requested shutdown or restarting a container. This signal is not sent when a container is being terminated for exceeding its memory limit, and may not be sent when a container is terminated because the user has run out of monthly execution hours. Currently most Archive Team software terminates immediately upon receiving a SIGTERM, so it may make sense to adjust the software to try to complete items within 30 seconds when it receives a SIGTERM signal.

Automatic Restart upon Application Exit

If the process that is started with the container exits, the container is restarted using a capped exponentially-increasing delay. The only way to prevent the container from being restarted is to scale down the application from the Heroku website or from the Heroku API (if the user provides their API token and app name to the container, one could write a script to use the API to scale the container down from within the container).

Multiple Accounts

There is a policy against using automation to create multiple accounts, however a policy against creating multiple accounts in general has not been found.

Deploying to Heroku

Heroku can use language-specific buildpacks or can deploy Docker images. Normally this is done via the command line, but it can also be done via the web browser if specific files are in your GitHub repo.

Required files for browser deployability of Docker images

app.json

Specifies what to display on the deployment webpage. See app.json Schema for more details Minimum properties are:

{
  "name": "Whatever you want to display",
  "formation": {
    "worker": {
      "quantity": 1,
      "size": "free"
    }
  },
  "stack": "container"
}

Dockerfile

This can be a project Dockerfile or if housed in a separate repo from the main project it can simply contain:

FROM [whatever docker image address]

heroku.yml

Should contain the following. Additional documentation is available for more options.

build:
  docker:
    worker: Dockerfile

Deploy link

From your repo, link to https://heroku.com/deploy (referrer is used to determine what to deploy). Or link to https://heroku.com/deploy?template=[GitHub repo link]. An image badge can optionally be used.

Note that Git submodules are not supported when deploying via button.

Examples

Tech234a has an unofficial GitHub repository that can be used to deploy the Archive Team Warrior to Heroku without leaving the web browser. Limitations include: inability to access the web interface (configure the Warrior using environment variables in your Heroku app settings), inability to see logs (aside from very minimal project download/installation logs), inability to log into the running container, and lack of filesystem persistence (this typically doesn't matter).

Recommended Enhancements to Archive Team Software to Fully Support Heroku

  • Ensure all Warrior logs are logged to console.
  • Add app.json, heroku.yml, and deploy button to Warrior repo.
  • Handle SIGTERM in a more efficient way: stop requesting new items and try to complete any existing items within the 30-second timeframe.
  • If the default project is selected and the default project can't be installed/can't run due to missing dependencies, try all current projects in order until one that works is found.
  • Add way to update key dependencies within Warrior without redeploying Docker image (wget-AT, etc.). A simple shell script would suffice.
  • Enable a port forwarding service for accessing the Warrior web interface or switch to a web dyno, listen on the Heroku-specified port, return 200 for /wakemydyno.txt, and use http://wakemydyno.com/ to keep the Heroku instance active/alive.
  • Add a Heroku API integration to the Warrior for updating configuration environment variables and shutting down/restarting the container.
  • Add a bandwidth-limiting feature to the Warrior to prevent exceeding the 2TB monthly download/upload limit (for unverified accounts this would be about 3.4GB/hour).