Niconico

From Archiveteam
Jump to navigation Jump to search
Niconico
Niconico logo
Wwwnicovideojp-full.png
URL https://www.nicovideo.jp[IAWcite.todayMemWeb]
Status Online!
Archiving status Partially saved
Archiving type Unknown
Project source niconico-grab
Project tracker niconico
IRC channel #niconino (on hackint)
Data[how to use] archiveteam_niconico

Niconico (http://www.nicovideo.jp/), formerly known as Nico Nico Douga is a Japanese video sharing site similar to Youtube managed by Niwango, a subsidiary of Dwango. As of December 2014, Niconico is the ninth most visited site in Japan. It has a unique comment system where the comments are overlaid directly over the video, synced to a specific video playback time. The site is known for its anime related content and memes.

Forced 2-factor login on every login began in February 2022 (see [1]).

Spinoff sites

Scripts

  • yt-dlp can download from many different sites, including Nico Nico
  • nicovideo-dl is a small command-line program to download videos from www.nicovideo.jp. It is implemented in Python. It is platform-independent and should work on Unix/Windows/Mac.It seems to be unavailable now.
  • nndownload works on Douga, Seiga, and Namahousou.

There are several other generic scripts and browser extension that can download videos from video sharing sites that will work with Niconico, but they can't download videos with the comments overlaid over the video.

Recommended way to archive niconico videos using nicovideo-dl

This method will require you to have an account registered on niconico. To get the highest quality video possible, you will need either need to buy a premium subscription (540 yen a month) or only download videos outside of the peak times (7pm - 2am JST). One way to tell if you're getting the higher quality download is that they usually have a .mp4 extension, whereas lower-quality downloads result in a .flv file.

nicovideo-dl --username "YOUR-USERNAME" --password "YOUR-PASSWORD" --get-comment --raw-comment --title http://www.nicovideo.jp/watch/sm20338084

This will grab the video and the comment XML file. The XML file can be converted into an ASS subtitle file compatible with most video players using the danmaku2ass script by m13253.

Recommended way to archive niconico videos using yt-dlp

yt-dlp --write-info-json --get-comments --write-sub --all-subs --write-thumbnail --username <USERNAME> --password <PASSWORD> https://www.nicovideo.jp/watch/sm21347640

Explanation for the arguments:

  • "--write-info-json" creates a "filename.info.json" file that contains all metadata for the video.
  • "--get-comments" informs yt-dlp that it should download comment data, and stores it in the info.json file
  • "--write-sub --all-subs" converts the above comment/danmaku data into .ASS subtitle files, for all 3 comment regions (English, Japanese and Chinese)
  • "--write-thumbnail" writes the thumbnail file to disk

If you are not opposed to editing the file container itself, then there are additional useful arguments:

  • "--add-metadata" embeds some metadata into the video file such that they show up in video players
  • "--embed-thumbnail --embed-subs" embed thumbnails and subtitle files into the video file respectively
  • "--remux mkv" remuxes the video into an MKV file, which is required for subtitles to be embedded and can handle more exotic Niconico stream formats than MP4. Does not work with some .swf file sources.

Similar to the above method, with a free account or with no account at all, there is a time range every day where downloading high-quality versions of videos is not available. However there are some nuances when downloading the highest quality videos.

If the video is sufficiently old enough (always prior to December 2016 and sometimes before June 2018), then the "smile_high" format is the highest quality as it is the source upload video. This format typically contains .swf, .flv or .mp4 files. Otherwise, the DMC formats are the highest quality. (smile_low is always the worst quality)

An indepth discussion of Niconico format qualities can be found here.

yt-dlp tries to automatically determine the best quality via these rules and other quality factors such as bitrate, however if you wish to examine yourself you can pass "-F" as an option and see the format selection yourself. Here is the format selection for the example video:

format code                extension  resolution note
h264_360p_low-aac_64kbps   mp4        640x360    DMC Low-Quality  364k , h264@ 300k, aac  @ 64k
h264_360p_low-aac_192kbps  mp4        640x360    DMC Low-Quality  492k , h264@ 300k, aac  @192k
h264_360p-aac_64kbps       mp4        640x360    DMC 360p  664k , h264@ 600k, aac  @ 64k
h264_360p-aac_192kbps      mp4        640x360    DMC 360p  792k , h264@ 600k, aac  @192k
h264_480p-aac_64kbps       mp4        854x480    DMC 480p 1664k , h264@1600k, aac  @ 64k
h264_480p-aac_192kbps      mp4        854x480    DMC 480p 1792k , h264@1600k, aac  @192k
smile_high                 mp4        864x486    High quality smile video 3419k , mp4 container, h264@3226k, aac  @189k, 94.59MiB (best)

Nico Nico Douga

A video sharing website and the core of the Nico Nico family.

Comments flow from right-to-left, an iconic feature of the website and called danmaku (弾幕). Comments can use a scripting language (ニコスクリプト Nicoscript). It was partially crippled in the 2020 transition away from Flash. Comment artisans (コメント職人) can produce particularly artful and extravagant ASCII art comments. Comment filters can replace keywords. Video authors can set their own video comments (投稿者コメント) that do not move into archives.

From February 28, 2018 an account is no longer required to watch videos.

Due to the limited server capacity, Niconico restricts access to the website to free users at peak times (7 p.m. to 2 a.m.) by forcing them to stream lower quality versions of videos or blocking access entirely to newly registered users. Economy mode was abolished in 2024.[1]

An English-language version was added on October 17, 2012, replacing the Niconico.com website, featuring a new player and translation tools allowing users to translate video descriptions into English or Chinese. The English and Chinese interface localizations were abolished in 2023[2]; the language switch remains, preserving video metadata translations and language-segregated video tags and comments.

Niconico formerly allowed upload of SWF videos; these are the videos whose ID begins with "nm" instead of "sm". A number of videos exploited this to implement strange gimmicks ([2]). An HTML5 player was trialled beginning in 2016 and made default across the site in 2017, replacing the old Flash player. After April 2018, all SWFs were converted to MP4. ([3]) Whether the original SWF is still accessible or not is unclear.

As of spring 2021, Niconico will begin deleting metadata and comments from deleted videos (which were previously visible to logged-in users).At the same time, the "past log," a feature that displays past comments that had been hidden due to restrictions on the number of comments that can be displayed, will also be discontinued with the renewal of the comment server, regardless of the availability of the video.[3] Most comments were archived (over 95%) and all metadata saved.

Under each video is a "marketplace" (ニコニコ市場 Nico Nico Ichiba) where products from Amazon and the like can be attached by users, often facetiously.

Third-party extraction services

3rd-party metadata archival websites:

Audio extraction websites (all defunct):

Overview of Video IDs

A video ID is a number given to each video, consisting of a two-letter alphabetic string representing the video site where the video was posted or the content holder providing the video, and a unique sequential number unique to the video itself. In short, it is the http://www.nicovideo.jp/watch/aa9999 part of the video URL. The following is a list of the types of videos.

IDs of videos that can be posted by users

  • sm - SMILEVIDEO videos. Can be attached to almost any video.
  • nm - Videos posted with official Nico Nico Video editing software (such as Nico Nico Movie Maker).

IDs of videos that could be posted in the past

These things are not linked automatically. Containing the metadata of the video.

  • am - Videos uploaded to "AmebaVision", which was discontinued in October 2007, making it impossible to watch videos.
  • fz - Videos uploaded to "Photozo", discontinued in May 2008, but still available for viewing.
  • ut - A video uploaded to "Youtube". Cannot be viewed now.
  • dm - A video uploaded to "DailyMotion", which was used by niconico.com.

Official Videos

  • ax - "avex" official
  • ca - "Cho! Animelo", "Nyoko Nyoko Video"
  • cd - (unknown)
  • cw - "Callawood Video"
  • fx - "MTV official", not supposed to be posted to SMILEVIDEO
  • ig - "i*gura video exit_nicochannel"
  • na - "Livedoor net animation"
  • om - "Otome"
  • sd - Basically, it's still used for development purposes by the management.
  • sk - "spike NicoNico Channel exit_nicochannel"
  • yk - "YuriChannel video"
  • yo - "Yoshi Yoshi video"
  • za - "NicoNicoNico Anime Channel"
  • zb - "NicoRadio&TV"
  • zc - "NicoAni News"
  • zd - "ai sp@ce"
  • ze - "Niji Audio Visual Room"
  • nl - NicoNico Live Broadcast
  • so - SmileOfficial. the distribution of dedicated identifiers for official videos such as the above has been discontinued, and all official video identifiers have been unified to this.

If you watch official videos via NicoNico Channel, you will be assigned a different ID for the channel video, so you may not see these videos as rarely.

Blomaga

NicoNico Blomaga[IAWcite.todayMemWeb] is NicoNico's blog platform. It was scheduled to shut down October 7, 2021[4] (and did go down then, but oddly enough seemingly at midnight Pacific time, instead of Japan time). A grab script was mostly written but never run; instead, a series of nico.ms links (see below) were generated and run through ArchiveBot. From IDs going to about 2 million, about 1.2 million were saved before the site shut down. The rest were run anyhow, in order to try to capture official (business account) blogs not included in the shutdown, on the grounds that these are probably going down soon anyhow.

As of the time of writing, a search of the ArchiveBot viewer for "niconico" results in an accurate listing of jobs. As almost always, the easy way to access the data is through the Wayback Machine.

Site Structure

Blomaga is divided into blogs (i.e. users), which contain posts. Blogs are identified by an alphanumeric ID (distinct from a more customizable blog title), whereas posts are globally serially numbered, with the number generally prefixed by the string "ar". https://ch.nicovideo.jp/article/ar[number] and https://nico.ms/ar[number] seem to redirect to posts with that ID. More work is needed to determine correspondence between blogs and video channels, and whether this may serve as another means of discovery.

Nico Nico Seiga

For hosting static artistic works, similar to Pixiv or Deviantart. Login is required to view works in full quality.

Comments do not flow across the image.

Nico Nico Shunga

The R-15 corner of Seiga. けしからん性力図 is a user-created heatmap of where the most sexual part of the image is. ([11])

Nico Nico Manga

([12]) A corner of Seiga hosting both user-created (ユーザー作品) and commercial (公式作品) comics. Each comic can hold a maximum of 100 chapters. Login is required to view past the first page of a chapter.

Chapters of commercial comics are usually available for free for one month, without geographic restriction, after which it is removed from the website.

Danmaku comments flow from right-to-left over the image.

Nico Nico Pedia

An encyclopedia website. It is a valuable source for Niconico culture, and along with Pixiv Encyclopedia, a valuable source for various trends.

As explained on the front page, articles are generally only editable by premium users. Articles for specific videos can also be edited by the video uploader, users can edit their own user pages, and community articles can be edited by community members.

Users can attach oekaki drawings (お絵カキコ oekakiko) or chiptune MML music (ピコカキコ pikokakiko) to article comments. The latter is at risk of JASRAC copyright takedowns.

Until December 2020, Nico Nico Pedia used a Flash-based Oekaki tool ([13]). The drawing process of posted drawings (お絵カキコ Oekakiko) could be replayed. The chiptune player was also Flash-based, but this writer believes nothing was lost.

Lists of latest posts of various types are accessible from the front page.

NicoNi-Commons

A commons website for content like free background music and 3D models. User ID numbers are different on Commons but still linked to a user's primary ID.

NicoNi-Community

Videos, seiga illustrations, and livestreams can be registered with a community. Communities have textboards (掲示板) that are only accessible to members.

Videos can be only accessible to members (https://vocadb.net/T/7446), example https://www.nicovideo.jp/watch/sm13929471.

Game Atsumaru

Game Atsumaru (ゲームアツマール)
URL https://game.nicovideo.jp/atsumaru/
Status Offline
Archiving status Partially saved
Archiving type ArchiveBot, other
IRC channel #archiveteam-bs (on hackint)
Data[how to use] job:3tt38940863qg137qhybq2xbe
job:bsx1a496u5w4cblcwmq5lyjq4

Game Atsumaru (ゲームアツマール) is Niconico's web game hosting website. Originally launched as RPG Atsumaru, it was the officially supported game upload platform for RPG Maker MV, and later added support for RPG Maker MZ and Akashic Engine. Since non-RPG games can be made using RPG Maker, the site was renamed Game Atsumaru in 2021.

In 2021 support was added for RPG Maker 2000/2003 using EasyRPG as an interpreter.

Danmaku comments are supported; they flow from right-to-left during the game.

On 20 December 2022, it was announced that the Game Atsumaru platform would be shutting down on 28 June 2023.[5] Uploaders have been given the ability to download their games. A new game hosting platform Nico Nama Game (ニコ生ゲーム) is scheduled to be be launched on 19 April 2023, at which point uploads to Game Atsumaru will be stopped.

After about 12pm JST, June 28, 2023, the entire https://game.nicovideo.jp/ will be redirected to https://blog.nicovideo.jp/niconews/194994.html .

Archiving status

Game homepages are archived by ArchiveBot.

There's a repo at https://github.com/yts98/game-atsumaru-discovery to discover as many static resources as possible. Volunteers from Archive Team and Save The Web Project joined this manual project.

  • Most of game frameworks lazily load resources, so staying on the page for a long time does not guarantee that all resources will be found. A difficult approach would be to categorize games by framework and then group them for static analysis.
    • Without archiving all the dynamically loaded data/[A-Za-z]+\.json and data/Map[0-9]{3}\.json, over 5989 games made with RPG Maker MV and over 1332 games made with RPG Maker MZ would be completely unplayable.
  • Danmaku comments and scoreboards are event-triggered, so it's better to check for:
    • possible sceneName (strings customized by the game author, such as __title, __gameover, map1) to archive danmaku comments.
    • boardId (1~10 or 1~30) to archive scoreboards.

Site structure

URLs are of the format https://game.nicovideo.jp/atsumaru/games/gm[number], where gm[number] is the gameId (up to approximately 28500 as of 2022-12-20, up to 29754 as of 2023-04-19). nico.ms short links work, as in https://nico.ms/gm[number]

As of June 24, 2023, 9490 games are publicly listed, 21 of which must be paid channel members to play; others were unlisted, private, or removed.

Unlisted games are of the format https://game.nicovideo.jp/atsumaru/games/gm29721?key=f45a1f70c172. They can be discovered by googling site:https://game.nicovideo.jp/atsumaru/games/ inurl:key or searching Wayback CDX. As of June 24, 2023, a total of 307 unlisted games have been found.[6]

Game loading mechanism

A game is actually an HTML with some static resources (JavaScript, CSS, WASM, images, audio), but the actual URL of the game is not embedded in the aforementioned page.

The ReactJS app will call the API method createPlayTicketV2() to get a ticket with a gameUrl in the format https://resource.game.nicovideo.jp/games/:gameId/:version/index.html and open a session that lasts 1800 seconds. If a client request game resources without a valid ticket, the server would simply respond with an HTTP 404.

curl commands 
curl -X GET -A "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0" \
     -H "X-Frontend-Id: 39" -H "X-Frontend-Version: 3.464.0" \
     -H "Origin: https://game.nicovideo.jp" -H "Referer: https://game.nicovideo.jp/" \
     "https://api.game.nicovideo.jp/v1/rpgtkool/games/gm9482/permission-info.json?sandbox=0"

curl -X POST -A "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0" \
     -H "X-Frontend-Id: 39" -H "X-Frontend-Version: 3.464.0" -H "X-Request-With: https://game.nicovideo.jp" \
     -H "Origin: https://game.nicovideo.jp" -H "Referer: https://game.nicovideo.jp/" -F "=" \
     -c "cookie.txt" "https://api.game.nicovideo.jp/v1/rpgtkool/games/gm9482/play-tickets.json?sandbox=0&wipAccessKey"

curl -X GET -A "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0" \
     -H "Referer: https://resource.game.nicovideo.jp/core/player/index.html?3f452cce70" \
     -b "cookie.txt" "https://resource.game.nicovideo.jp/games/gm9482/1147/index.html"

Unlisted games can only be accessed by appending a valid key to the URL:

curl commands 
curl -X GET -A "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0" \
     -H "X-Frontend-Id: 39" -H "X-Frontend-Version: 3.464.0" \
     -H "Origin: https://game.nicovideo.jp" -H "Referer: https://game.nicovideo.jp/" \
     "https://api.game.nicovideo.jp/v1/rpgtkool/games/gm289/permission-info.json?sandbox=0&wipAccessKey=b0495694f39e"

curl -X POST -A "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0" \
     -H "X-Frontend-Id: 39" -H "X-Frontend-Version: 3.464.0" -H "X-Request-With: https://game.nicovideo.jp" \
     -H "Origin: https://game.nicovideo.jp" -H "Referer: https://game.nicovideo.jp/" -F "=" \
     -c "cookie.txt" "https://api.game.nicovideo.jp/v1/rpgtkool/games/gm289/play-tickets.json?sandbox=0&wipAccessKey=b0495694f39e"

curl -X GET -A "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0" \
     -H "Referer: https://resource.game.nicovideo.jp/core/player/index.html?3f452cce70" \
     -b "cookie.txt" "https://resource.game.nicovideo.jp/games/gm289/12/index.html"

When window.origin is in the constant list validOrigins, the app loads the game player in the iframe, and further loads the gameUrl in the game player's iframe, providing an interactive API[7] object window.RPGAtsumaru for the game.

Vital signs

Seems stable, though premium user counts are still declining. There was a scare in April 2020 when a clause relating to "service ending" was added to their Terms of Service, leading to rumours of the site potentially shutting down, but it turned out to be due to a change in legal requirements.

Activity is declining, its place in Japanese society being overtaken by YouTube.

External links

References