Difference between revisions of "SourceForge"

From Archiveteam
Jump to navigation Jump to search
(oops, missed one)
 
(15 intermediate revisions by 9 users not shown)
Line 1: Line 1:
{{Infobox project
{{Infobox project
| title = SourceForge
| image = SourceForge.png
| image = SourceForge.png
| description =
| URL = {{url|https://sourceforge.net/}}
| URL = {{url|1=http://sourceforge.net/|2=sourceforge.net}}
| project_status = {{online}}
| project_status = {{online}}
| archiving_status = '''Paused, Awaiting SF Staff Reply'''
| archiving_status = {{notsavedyet}}
| source = [https://github.com/ArchiveTeam/sourceforge-grab sourceforge-grab], [https://github.com/ArchiveTeam/sourceforge-grab-rsync sourceforge-grab-rsync]
| source = [https://github.com/ArchiveTeam/sourceforge-grab sourceforge-grab], [https://github.com/ArchiveTeam/sourceforge-grab-rsync sourceforge-grab-rsync]
| tracker = [http://tracker.archiveteam.org/sourceforge sourceforge], [http://tracker.archiveteam.org/sourceforgersync sourceforgersync]
| tracker = [https://tracker.archiveteam.org/sourceforge sourceforge], [https://tracker.archiveteam.org/sourceforgersync sourceforgersync]
| irc = coldstorage
| irc = sourceforget
}}
}}


Line 15: Line 13:
It's really old, ad supported, adware supported. And yet, it is still alive.
It's really old, ad supported, adware supported. And yet, it is still alive.


It hosts code migrated from [[BerliOS]]<ref>https://joinup.ec.europa.eu/news/german-open-source-development-site-berlios-joins-sourceforge</ref> which shut down.
It hosts code migrated from [[BerliOS]]<ref>{{URL|https://joinup.ec.europa.eu/collection/open-source-observatory-osor/news/german-open-source-developmen}}</ref> which shut down.


== Shutdown? ==
== Vital signs ==


=== 2015: Removal of FRS Area ===
=== FRS Area removal (2015) ===


<pre style="white-space:pre-wrap">
<pre style="white-space:pre-wrap">
Line 31: Line 29:
SourceForge.net Support
SourceForge.net Support
sfnet_ops@slashdotmedia.com
sfnet_ops@slashdotmedia.com
https://sourceforge.net/support</pre><ref>https://i.imgur.com/jll3l5V.png</ref>
https://sourceforge.net/support</pre><ref>{{URL|https://i.imgur.com/jll3l5V.png}}</ref>


=== 2015: Admins hijacking projects to add more adware ===
=== Admins hijacking projects to add more adware (2015) ===


http://lwn.net/SubscriberLink/646118/f8f6483b64fdafb9/
{{URL|https://lwn.net/Articles/646118/}}


== Site Structure ==  
== Site structure ==  


* 444,202 project URLs found: https://github.com/marcroberts/archiveteam-sourceforge-lister/blob/master/projects-sorted.txt
* 444,202 project URLs found: {{URL|https://github.com/marcroberts/archiveteam-sourceforge-lister/blob/master/projects-sorted.txt}}


Download files can be found on public ftp mirrors, priority on the rest of the site then download files last? e.g. http://www.mirrorservice.org/sites/ftp.sourceforge.net/
Download files can be found on public ftp mirrors, priority on the rest of the site then download files last? e.g. {{URL|https://www.mirrorservice.org/sites/ftp.sourceforge.net/}}


CVS/svn/git/hg/bzr repositories should be a priority; many projects do not have their source code on the ftp mirrors.
CVS/svn/git/hg/bzr repositories should be a priority; many projects do not have their source code on the ftp mirrors.


The main API is documented here: http://sourceforge.net/p/forge/documentation/Allura%20API/ and allows unauthenticated access to most services.  It also can indicate what revision control system is used.
The main API is documented here: {{URL|https://sourceforge.net/p/forge/documentation/Allura%20API/}} and allows unauthenticated access to most services.  It also can indicate what revision control system is used.


Appropriate tools, (such as git clone -m and svnrdump) can be used to backup, but SF suggests using rsync regardless of the actual revision control system used.
Appropriate tools, (such as <tt>git clone -m</tt> and <tt>svnrdump</tt>) can be used to backup, but SF suggests using rsync regardless of the actual revision control system used.


* Some projects have subdomain sites. Ex: http://supertuxkart.sourceforge.net/ Many can be listed by using the project API as an "external_homepage".
* Some projects have subdomain sites on either <tt>sourceforge.net</tt> or <tt>sourceforge.io</tt>. Ex: {{URL|http://supertuxkart.sourceforge.net/}}. Many can be listed by using the project API as an "external_homepage". These are not necessarily static sites; some of these use [[MediaWiki]] or other other software (e.g. {{URL|https://wala.sourceforge.net/}} or {{URL|https://nsis.sourceforge.io/}})


== How can I help? ==
== 2015 archival ==


There are two projects: one that grabs the web content and a copy of the binaries, and another that grabs the sourcecode repositories via rsync.
On June 17, 2015, ArchiveTeam started two simultaneous grabbing process: one for web-based content and binaries, and one for rsync-able source code repositories. Shortly afterwards, someone claiming to be a SourceForge staff member, told us to stop and first contact their representative.


For both, you can choose selecting the project in the [[Warrior]] appliance (only one of them), or set up and run the script(s) manually.
<div style="width:100%">
<pre>
jún 18 22:08:45 <burley-sf> FYI: I just blocked your archive client
jún 18 22:09:05 <JRWR>      oh?
jún 18 22:09:07 <burley-sf> it's not following robots.txt, and hitting recursive deep dives
jún 18 22:09:18 <JRWR>      oh my
jún 18 22:09:24 <arkiver>  burley-sf: We're currently trying to archive the software on your website
jún 18 22:09:26 <burley-sf> I'll also be killing the rsync's here soon, you are going too heavy on this
jún 18 22:09:37 <burley-sf> I understand, and I am OK with that -- but not the way you are doing it
jún 18 22:09:58 <arkiver>  burley-sf: What is your limit?
jún 18 22:09:59 <burley-sf> I suggest you stop, so I don't have to block the IPs for rsync
jún 18 22:10:06 <burley-sf> and reach out to our community guy
jún 18 22:10:14 <burley-sf> gimme min and I'll give you an email address
jún 18 22:10:24 <achip>    rsync is paused
jún 18 22:10:38 <arkiver>  burley-sf: thank you
jún 18 22:11:05 <burley-sf> rgaloppini@slashdotmedia.com
[...]
jún 18 22:36:13 <burley-sf> So reach out to Roberto at the address above and then I am sure we can sort something out that doesn't cause impact to the other users
jún 18 22:37:35 <burley-sf> And if you need to reach me for some reason -- david@sourceforge.net
</pre>
</div>


=== Web grab ===
We attempted to contact them but got no reply.


'''Warrior:''' SourceForge
== Developer Web removal (2025) ==


'''Script:''' http://github.com/ArchiveTeam/sourceforge-grab
{{Infobox project
 
| title = SourceForge Developer Web Services
=== Code rsync ===
| URL = https://sourceforge.net/
 
| project_status = {{closing}}
'''Warrior:''' SourceForge Rsync
| archiving_status = {{upcoming}}
 
| archiving_type = DPoS
'''Script:''' http://github.com/ArchiveTeam/sourceforge-grab-rsync
| irc = sourceforget
 
| source =
'''IMPORTANT''': in case of the rsync project, an item can be even several gigabytes in size! By default, item size accepted by the script/Warrior is limited to 5 GB. If you have orders of magnitude more space, you can bypass this limit (what's more, please, do), see the script README how. '''Note:''' count with twice the size of an item (that is, the downloaded copy plus the tar to be uploaded both sit on your HDD until the item finishes). Also multiply it by the concurrency level (Warrior default: 2).
| tracker =  
 
| data =  
'''Note:''' rsync download processes are limited, only one can run at the same time (to prevent banning from SourceForge's side).
}}
 
=== General info for script runners ===
Read the instructions (README) of the corresponding repository.


{| class="mw-collapsible mw-collapsed" style="text-align:left;"
On {{datetime|2025-08-09}}, SourceForge announced in a blog post that the Developer Web Service, the web hosting service for users, will be shutting down on {{datetime|2025-10-10}}.<ref>{{URL|https://sourceforge.net/blog/sunsetting-developer-web-user-web/}}</ref>
! Some additional information
|-
| Don't forget to replace YOURNICKHERE with your nickname.


The number after <code>--concurrent</code> determines how many threads run at the same time. You can increase this number if your resources (RAM, CPU, HDD, bandwidth) are sufficient. However, if you constantly see messages about rate limiting, there is no need to increase the concurrency.
<blockquote>


If you want to stop the script, please do it gracefully if possible. To do so, create an empty file named '''STOP''' in the folder of the script (terminal command: <code>touch STOP</code>). The script finishes the current item(s) and stops only after that. (If you kill the script immediately, the items get broken, and they will need to be reassigned to another user.) – Before starting the script again, don't forget to remove the STOP file.
=== Sunsetting Developer Web (User Web) ===


If you see "Project code is out of date", kill the script, go to its folder and issue <code>git pull REPOSITORY</code>, where REPOSITORY stands for the URL of either the <code>sourceforge-grab</code> or the <code>sourceforge-grab-rsync</code> repository, see above. After the updating has finished, re-launch the script.
SourceForge will be sunsetting developer web hosting for user accounts (unrelated to project web hosting) in 60 days on October 10th, 2025.
|}


=== Donating to the Internet Archive ===
If you are using developer web hosting on your SourceForge user account, please make arrangements before October 10th, 2025. Project web hosting for projects will continue as is.


Content downloaded by the ArchiveTeam will be uploaded to the [[Internet Archive]], where it will be stored and be available – hopefully – forever. However, storing it costs thousands of dollars in the long run. So, if you can afford, please consider donating to the Internet Archive, so that this piece of history can be kept for us all. http://archive.org/donate
Please refer to our documentation if you need help accessing your files hosted on developer web hosting.


=== Do you like our cause? ===
</blockquote>


If you want to help in other projects, want to learn more about ArchiveTeam, or even help in development in general, navigate to the [[Main Page]] of this wiki, from there you can reach a lot of information. The Team consists of volunteers working on the projects in their free time, so helping hands (and resources) are always welcome.
{{CTA URL lists|regex=users.sf.net{{!}}users.sourceforge.net{{!}}sf.net/u/{{!}}sf.net/users/{{!}}sourceforge.net/u/{{!}}sourceforge.net/users/|level=3}}


== References ==
== References ==
Line 101: Line 112:


== External links ==
== External links ==
* {{url|1=http://sourceforge.net/|2=SourceForge}}
* {{url|https://sourceforge.net/|2=SourceForge}}


{{Navigation box}}
{{Navigation box}}


[[category:Code]]
[[category:Code]]

Latest revision as of 17:32, 30 September 2025

SourceForge
SourceForge.png
URL https://sourceforge.net/[IA•Wcite•.today•MemWeb]
Status Online!
Archiving status Not saved yet
Archiving type Unknown
Project source sourceforge-grab, sourceforge-grab-rsync
Project tracker sourceforge, sourceforgersync
IRC channel #sourceforget (on hackint)

SourceForge is a free software repository.

It's really old, ad supported, adware supported. And yet, it is still alive.

It hosts code migrated from BerliOS[1] which shut down.

Vital signs

FRS Area removal (2015)

Hello,
You have been identified as having saved files in you user FRS profile area (/home/pfs/<username>. We are planning on removing this
area for user accounts on March 17th 2015. We wanted to give you the opportunity to move your data to a new location before we
remove the data. Here is a link that should help you with moving your data:
https://sourceforge.net/p/forge/documentation/SFTP/
If you need any help please contact us.
Thanks
SourceForge.net Support
sfnet_ops@slashdotmedia.com
https://sourceforge.net/support

[2]

Admins hijacking projects to add more adware (2015)

https://lwn.net/Articles/646118/[IA•Wcite•.today•MemWeb]

Site structure

Download files can be found on public ftp mirrors, priority on the rest of the site then download files last? e.g. https://www.mirrorservice.org/sites/ftp.sourceforge.net/[IA•Wcite•.today•MemWeb]

CVS/svn/git/hg/bzr repositories should be a priority; many projects do not have their source code on the ftp mirrors.

The main API is documented here: https://sourceforge.net/p/forge/documentation/Allura%20API/[IA•Wcite•.today•MemWeb] and allows unauthenticated access to most services. It also can indicate what revision control system is used.

Appropriate tools, (such as git clone -m and svnrdump) can be used to backup, but SF suggests using rsync regardless of the actual revision control system used.

2015 archival

On June 17, 2015, ArchiveTeam started two simultaneous grabbing process: one for web-based content and binaries, and one for rsync-able source code repositories. Shortly afterwards, someone claiming to be a SourceForge staff member, told us to stop and first contact their representative. 

jún 18 22:08:45 <burley-sf> FYI: I just blocked your archive client
jún 18 22:09:05 <JRWR>      oh?
jún 18 22:09:07 <burley-sf> it's not following robots.txt, and hitting recursive deep dives
jún 18 22:09:18 <JRWR>      oh my
jún 18 22:09:24 <arkiver>   burley-sf: We're currently trying to archive the software on your website
jún 18 22:09:26 <burley-sf> I'll also be killing the rsync's here soon, you are going too heavy on this
jún 18 22:09:37 <burley-sf> I understand, and I am OK with that -- but not the way you are doing it
jún 18 22:09:58 <arkiver>   burley-sf: What is your limit?
jún 18 22:09:59 <burley-sf> I suggest you stop, so I don't have to block the IPs for rsync
jún 18 22:10:06 <burley-sf> and reach out to our community guy
jún 18 22:10:14 <burley-sf> gimme min and I'll give you an email address
jún 18 22:10:24 <achip>     rsync is paused
jún 18 22:10:38 <arkiver>   burley-sf: thank you
jún 18 22:11:05 <burley-sf> rgaloppini@slashdotmedia.com
[...]
jún 18 22:36:13 <burley-sf> So reach out to Roberto at the address above and then I am sure we can sort something out that doesn't cause impact to the other users
jún 18 22:37:35 <burley-sf> And if you need to reach me for some reason -- david@sourceforge.net

We attempted to contact them but got no reply.

Developer Web removal (2025)

SourceForge Developer Web Services
URL https://sourceforge.net/
Status Closing
Archiving status Upcoming...
Archiving type DPoS
IRC channel #sourceforget (on hackint)

On 2025-08-09, SourceForge announced in a blog post that the Developer Web Service, the web hosting service for users, will be shutting down on 2025-10-10.[3]

Sunsetting Developer Web (User Web)

SourceForge will be sunsetting developer web hosting for user accounts (unrelated to project web hosting) in 60 days on October 10th, 2025.

If you are using developer web hosting on your SourceForge user account, please make arrangements before October 10th, 2025. Project web hosting for projects will continue as is.

Please refer to our documentation if you need help accessing your files hosted on developer web hosting.

How to help if you have lists of URLs

For other ArchiveTeam projects that can use this kind of help, see Projects requiring URL lists.

This project requires lists of URLs for content on the target website. If you have a source of URLs, please:

  1. Use the PCRE regular expression users.sf.net|users.sourceforge.net|sf.net/u/|sf.net/users/|sourceforge.net/u/|sourceforge.net/users/ for filtering.
    • Enable case-insensitive matching (e.g. grep's -i) to catch URLs with capitalization.
    • If using grep or similar, enable text matching (-a or --text) to catch URLs in files with apparent binary data.
    • Example command (GNU grep): grep -Pahoi 'users.sf.net|users.sourceforge.net|sf.net/u/|sf.net/users/|sourceforge.net/u/|sourceforge.net/users/' FILENAME FILENAME...
  2. If the output exceeds a few megabytes, compress it, preferably using zstd -10.
  3. Give the file a descriptive name and upload it to https://transfer.archivete.am/.
  4. Share the resulting URL in the project IRC channel.
    • If you wish your list to remain private, please get in touch with a channel op (e.g. arkiver or JustAnotherArchivist). Items generated from your list will still be processed publicly, but they will be mixed in with all other items and channel logs will not associate them with you.

References

  1. ↑ https://joinup.ec.europa.eu/collection/open-source-observatory-osor/news/german-open-source-developmen[IA•Wcite•.today•MemWeb]
  2. ↑ https://i.imgur.com/jll3l5V.png[IA•Wcite•.today•MemWeb]
  3. ↑ https://sourceforge.net/blog/sunsetting-developer-web-user-web/[IA•Wcite•.today•MemWeb]

External links

Retrieved from "https://wiki.archiveteam.org/index.php?title=SourceForge&oldid=57540"