A 2008 summary of site statistics, review of site improvements, and a look at groups and Greasefire. Want a t-shirts?
52,956,864 Pageviews in 2008

These page views are by real visitors (running JavaScript) and do not include downloads (scripts, images, or other static content), userscript updaters, ajax requests, spiders, and other bot traffic.
Over 1,400,000 dynamic requests per day

An average of 900 requests per minute (cpm is calls per minute) hit the rails code running the site (that is over half a billion dynamic requests per year). Pingdom lets me know when the server explodes (which happens at the worst possible times). The site has had less than 3 hours downtime over the last 3 months.
3.5TB downloaded, 0.5TB uploaded

The downstream bandwidth has averaged 971Kb per second for the year (1.4Mb per second since September!). If the site was hosted on Amazon's EC2, bandwidth would cost $445 for the year. I expect the bandwidth to double in 2009.
Constant battles with CPU

Red is BAD! To combat constant growth, there have been many rewrites and infratstucture improvements. Systems changes include sphinx for search, Phusion Passenger for app servers, Quantcast for public stats, and NewRelic for performance monitoring.
Site Improvements
I cannot stress enough that the help given by community members in the site forum has been invaluable. After a long day working on my startup, their contributions give me the energy to spend more time in front of a computer trying to keep up with them. I'm grateful for all levels of help, from reporting a simple bug, helping newbies in forums, to writing html, css, and userscripts to improve the site.
The forum system was rewritten several times: moving script comments to forums, dealing with performance issues, read/unread status rewrite, stylistic improvements, and more. There were 44,673 posts in 2008 (and that doesn't include all the deleted spam).
The site had several major stylistic changes. The major being a new site header and script page. The year also brought many stylistic improvements from members (suggested in the forum).
Reviews were added in late October as a way to help users find the best scripts. So far 702 have been written (with 331 written in December).
Instead of overwriting the data with each upload, storing older versions of scripts was implemented. This helps answer the question "what changed?" and allows the community to watch for funny business by malicious script authors.
There are 25,386 scripts on the site, of which 15,286 were uploaded/updated in 2008.
The Future (where is my jet pack?)
In 2009 I'm concentrating on features that help users sort through all those scripts.
Steve Krulewitz wrote an open source extension called Greasefire help users know when there are scripts on userscripts.org written for the page they are viewing. Join him in the greasefire forum to discuss this new feature.
I've been working on groups with the goal of making it easier to find, share and collaborate on scripts (the first version released yesterday). A group can be centered around a site (NeoPets, Travian, Orkut, ...), concepts (using xpath), or a place for people to collaborate. The initial version of groups include members, forums and script collections. As the feature matures I hope to feature groups on the site homepage.
I plan on moving the entire site to Amazon's scalable infrastucture as I'm hitting the edge of what a single box can handle.
Last, I'm debating having a "userscripts t-shirts" made. Would you be interested in buying one? (probably $20 plus shipping for a high quality shirts) Let me know if you have opinions on the design or would buy one.
I hope that 2009 is another great year!
You could comment on this post if you were logged in.
login to vote
I like the groups idea. T-shirts would be neat too. :)
login to vote
Great post, very interesting to read all that information.
You should make a time-line with all the great implementations you've made.
login to vote
You did great job on USo last year. You have my thank.
login to vote
Thank you for your dedication to the site
login to vote
Thanks for everything you've done and plan to do!
login to vote
I say the server rolls over and dies by March lol.
login to vote
I just got off the phone with softlayer - I just upgraded to a $530/mo server plan (was paying about half that at serverbeach, whose support isn't as good as softlayer) - I hope to move tomorrow.
Having one really good server seems better than a bunch of smaller servers for now, so maybe Amazon's EC2 next year...
login to vote
Provided you don't change the site structure dramatically, I bet the pageview count will be at 400,000 by the end of next year.
login to vote
Jesse, maybe you should put a paypal "Donate" button somewhere on the site so people can chip in and help you pay for the hosting costs.
login to vote
Sign by Danasoft - For Backgrounds and Layouts
login to vote
Congrats on your success through 2008 and my best hopes to you in the new year!
login to vote
Wow, you've been busy! I can't believe how much this place has grown since it first started *nostalgia*. Oh, and I second Yansky's idea for a Donate button. Maybe in the form of one of those "Buy Me a Cup of Coffee" buttons you see all the kids with blogs using these days.
login to vote
What about issues currently pending... (around 400 pages... with almost 20 script reports on each!!!) ???
login to vote
Yes, reporting issues seems pretty useless if you're not gonna do anything with the report
login to vote
Whatever happened to having scripts that contained "cookie" in them automatically unlisted. They would still be added to the "need to be reviewed" box but then it would help protect the unsuspecting user from installing all of those cookie grabbers that have popped up.
login to vote
@nungryscpro
there are plenty legit scripts that contain "cookie" and there are many ways to obfuscate calls to document.cookie. so if such a system as you described were put in place only the non-malicious scripts that have no reason to hide their intentions would be unlisted, and malicious scripts would still be listed, creating a false sense of security for users. I mean, how the hell do you suggest we prevent something similar to:
login to vote
I though the point of the reports were to use them to solve issues
login to vote
hey you all peoples ...plz help me
im having problem in my orkut profile
i hav installed all de scripts but i can't found a change in my profile
tell me how torun all dis...
wat to do...??? :(
login to vote
var first = "c";
// Hundreds of lines of code
var steal = document[first + "ookie"];
how to use dis/any scripts...plz tell me....
login to vote
@relish: Are you asking us to tell you how to create a cookie-stealing script?
login to vote
I understand there are legit scripts that contain "cookie" but that's how the sytem used to be set up. I had one of my scripts initially unlisted and had to get it listed by the staff all because I had to reference "cookie.gif"
I guess since they no longer wish to be bothered by reports, they've allowed cookie stealing script to get listed automatically. Under the old system there were 0 cookie grabbing scripts for keyword "neopets". Now there are 14 that have been uploaded within the last 6 weeks. (Half don't actually work because they were written badly.)
When a script is listed or updated, it should be easy enough to parse the code, not just reading it, but basically running it and then searching the resulting page for "document.cookie". I know there are also scripts that aren't malicious that access "document.cookie" but are there going to be that many that the staff will have to list manually? They already have it set up so that when you visit the source code, the script is parsed for syntax highlighting.
In the end, those 14 or so scripts have all been reported but all of them are still there. Even with a couple warnings in the comments the install counts rise. Someone needs to do something about the reports or set up a system that blocks them in the first place!
login to vote
@nungryscpro
First of all their is no staff. Jesse is the only active admin. This is a community driven site. It is up to those of us who understand javascript to be on the lookout and warn people in comments if a script is malicious and it is up to users to take the time to look at the comments to see if a script has been called malicious. Also the install count isn't entirely truthful. You need to install a script to raise its install count. Hell 4.66% of traffic a month is by those who have IE6.
How many scripts were there for neopets?
Yes there would. Jesse is just one man. He would have to sit in front of a computer all day listing scripts. There are literally thousands of legit scripts that use "document.cookie". Hell 3 out of my 16 scripts use it.
Highlighting is done on the client-side. Checking for
document.cookiewould have to be done on the server-side.The bottom line is that it is just totally impractical to manually approve every script that contains
document.cookie. What happens when someone uploads a new version? Do we have to wait for that to be approved before it is listed again? Let's not turn Userscripts.org into Addons.mozilla.orglogin to vote
There were 0 cookie grabbing scripts out of around 90.
As of right now, there are 18 out of 123. A little less than half of the scripts since the unlisted feature was removed have been cookie grabbers.
You should try using GM_setValue and GM_getValue instead. If you're worried about making the scripts work in browsers besides Firefox, greasekit emulates GM functions and there is Opera Userscript for Opera users.
Of course. If that can be written to work client side, think of all you can do server side.
As far as manually approving unlisted scripts... whatever happened to the idea of having a real community Reviews Discussion. There's a help file for unlisted scripts unlisted scripts. It says there to email admin. There are a lot of people that care about us.o and are willing to help with the workload.
There are currently 405 pages of harmful/spam scripts. How many will still be there next week?
login to vote
@nungryscpro,
you are correct that something needs to be done about this. And thank you sizzlemctwizzle for explaining that this is a community site.
I've been slow to expose the controls needed, since I'm afraid of getting the balance wrong... Instead of worrying about getting it perfect on the first shot I need to release a first cut so that improvements can be made.
Thank you both (and others before) for caring enough to help improve this situation!
login to vote
Please consider deleting/editing the basically useless script here:
http://userscripts.org/scripts/show/46036
It is basically spam because it shows up for every www.* site in Greasefire and makes greasefire much less useful. The review and discussion there also state this case. Thanks!
login to vote
What about implementing a function in this site that enables to user to vote whether a script is malicious and to show how many users voted the script as malicious (and how many as not malicious) and to show this (somehow highlighted) just below the "Install"-link?
This would help to prevent users from downloading propably malicious scripts.
Maybe there could also be a message-box that asks you if you really want to install a script if there were enough votes (f.e. more than 20) and more than half of the votes say the script is malicious.
login to vote
n-regen,
We are working on that right now. If you look there is now a "issues" tab that lets you vote and comment on a script's badness.
login to vote
Great new page updates, tags and layout improvements! Keep up the good work [it's much appreciated!]