For anyone still watching, the crawl's just completed. I'm processing stats now.

Currently looking at:

- Most mentioned states (may add territories).

- Most mentioned cities (330 or so largest US for now, may add selected global cities). .

- Most mentioned countries / country identifiers, including EU.

Also looking at submitters, submitted sites, associated votes and comments, and trends over time, though those may take a bit longer to complete.

Considering companies and universities as well.

Analysis is largely awk, grep, sort, uniq --- standard shell tool stuff. Amazingly powerful though really. Crawl via wget which performed wonderfully as usual. Thanks to HN/YC for not throttling too aggressively.

And, after looking again at HN's API and determining that front pages aren't represented in it ... I'm crawling the 5,939 front pages from 20 Feb 2006 to 25 May 2023 and scraping those to find mentions of states on the actual front pages.

Aggressive crawls result in 403 after a few fetches, so my current attempt has an apparently-adequate delay parameter set. It'll take a day or two for the crawl to complete. Presently up through 22 Sept 2007.

State-name matches (with some false positives) in that set:

  Delaware corporate franchise tax is due today (March 1st)  (March 1, 2007)
  An Early Stage Entrepreneur's Guide to New York City  (March 5, 2007)
  Web 2.0 Expo 2007, April 15-18, 2007, San Francisco, California  (March 10, 2007)
  Zillow Becomes Illegal in Arizona  (April 16, 2007)
  New York Times on the social news uproar  (May 3, 2007)
  What Silicon Valley Could Learn from Columbus, Ohio  (May 22, 2007)
  New York Times Will Lower Editorial Standards Online And Reduce Size Of Print Newspaper  (May 26, 2007)
  eHarmony sued in California for excluding gays  (June 1, 2007)
  New York Times versus Digg  (June 6, 2007)
  A Patent Lie - New York Times  (June 9, 2007)
  Interactive Colorado Nightlife Guide - "Social Nightworking"  (June 16, 2007)
  In the state of Hawaii, tax breaks make tech investments nearly risk-free?  (June 27, 2007)
  Top Web Apps in Arizona  (June 28, 2007)
  Millionaires Cash Out Of California  (July 16, 2007)
  C.E.O. Libraries Reveal Keys to Success - New York Times  (July 21, 2007)
  Pics of Web Company HQs in California - Digg, Facebook, OpenDNS..  (July 25, 2007)
  Hackers find serious problems in California voting machines  (July 30, 2007)
  A Mystery Solved: "Fake Steve" Blogger Comes Clean - New York Times  (Aug 5, 2007)
  Colorado Startup Scene  (Aug 5, 2007)
  New York Times Sees Sense: Paywall Comes Crashing Down  (Aug 7, 2007)
  Parsing Miss South Carolina's Statement  (Sept 1, 2007)
  Texas Startup Says It Has Batteries Beat  (Sept 5, 2007)
  Spider-like vessel hits New York waters - can cross the Atlantic on one load of diesel fuel  (Sept 8, 2007)
  New York Times Launches Facebook App  (Sept 12, 2007)
  Theory Girl by the University of Washington CSE Band [mp3]  (Sept 15, 2007)

Note that seven of the "New York" mentions are for "The New York Times".

Dates covered: 2007-2-20 -- 2023-5-25

Total stories: 178,072 (Should match 5,939 days * 30 stories, seems to come a bit short with 98 fewer stories than expected.)

- States mentioned: 50 (mentions: 1,344)

- Cities mentioned: 109 (checked against list of 330 largest US cities, plus a few additions of my own).

- Sites submitted: 52,687

Top 20 states by raw string-match (this is adjusted below for some overstatements):

   1.  new york:         316
   2.  california:       308
   3.  texas:             98
   4.  washington:        73
   5.  colorado:          38
   6.  florida:           38
   7.  arizona:           28
   8.  georgia:           24
   9.  utah:              24
  10.  kansas:            23
  11.  virginia:          23
  12.  oregon:            22
  13.  michigan:          19
  14.  minnesota:         16
  15.  north carolina:    16
  16.  ohio:              16
  17.  alaska:            15
  18.  indiana:           15
  19.  hawaii:            14
  20.  maine:             14

Confounding factors include other terminology, other name matches:

"New York" frequently appears in reference to newspapers or cities:

  new york times:            98
  new york city:             55

So adjusted NY score: 163

"Washington" is even harder to disambiguate, as it may refer to the state, city (DC), as a topnym for the US / US Government, etc., It's challenging to identify these textually, though the newspaper and monument mentions can be identified:

  washington post:           16
  washington times:           1
  washington monument:        1

Adjusted WA score: 55. And is probably lower.

California is often mentioned by toponym (e.g., "silicon valley", region (bay area), or specific cities (SF, LA, etc.). I'm NOT going to adjust for these but do note the following major occurences:

  silicon valley:           399
  bay area:                  65
  california ave:             2

There are a few other more minor examples such as "kansas city" (confounding "kansas"), "oklahoma city", and "iowa city", though those don't affect top results.

Adjusted top-5 state rankings:

  1.  california:       308
  2.  new york:         163
  3.  texas:             98
  4.  washington:        55
  5.  colorado:          38

So, answering the original question, Colorado is the fifth-most-loved front-page state.

The 10 least-represented states:

  41.  new mexico:         6
  42.  alabama:            5
  43.  idaho:              5
  44.  north dakota:       5
  45.  connecticut:        4
  46.  nebraska:           4
  47.  new hampshire:      4
  48.  rhode island:       3
  49.  south dakota:       3
  50.  south carolina:     1

Top-10 US city-name mentions (see confounding factors above):

   1. new york:         316
   2. san francisco:    192
   3. boston:           117
   4. seattle:           87
   5. hollywood:         76
   5. tempe:             69
   6. berkeley:          63
   7. chicago:           62
   8. cambridge:         50
   9. detroit:           41
  10. los angeles:       30

(This excludes the NYC burough of Manhattan, which gets 36 mentions itself, nudging out LA.)

I've also got a list of most-featured front-page sites, with the ten most frequent being: nytimes.com, techcrunch.com, arstechnica.com, bloomberg.com, wired.com, wsj.com, youtube.com, wikipedia.org, BBC (as both bbc.com and bbc.co.uk), and theguardian.com. The mean votes and comments for each are interesting, with Bloomberg topping by both average vote (255.46) and comments (178.04), from the top-10. (There are obviously sites with much higher mean scores but less frequent appearance, including singletons).

I've broken submitters by freqency, points, and comments as well, with 43,745 submitters accounting for the stories analised. The long-absent nickb tops front-page appearances at 1,322, followed by ingve (1,177) and tosh (679). I've apparently landed 60 front-page submissions averaging 329.00 points and 147.52 comments (both fairly respectable --- I'm apparently not submitting total tosh).

I'm still updating my code and checking data (and found a couple of glitches writing out the above, e.g., whatever I'd done to tabulate city mentions didn't work). Data exist presently in flatfiles, I'm thinking of dumping to sqlite for possibly easier/faster analysis.

A rough and preliminary look at countries represented in front-page titles, again with numerous issues:

   1.  US:  1350
   2.  U.S.:  1073
   3.  China:  634
   4.  Japan:  526
   5.  India:  477
   6.  UK:  288
   7.  EU:  225
   8.  Russia:  221
   9.  Germany:  165
  10.  Canada:  162
  11.  Australia:  157
  12.  Korea:  140
  13.  France:  116
  14.  Iran:  91
  15.  Dutch:  80
  16.  United States:  75
  17.  Brazil:  69
  18.  North Korea:  69
  19.  Sweden:  68
  20.  Cuba:  67
  21.  Egypt:  60
  22.  Norway:  60
  23.  USA:  59
  24.  E.U.:  54
  25.  Israel:  53
  26.  Iceland:  52
  27.  New Zealand:  47
  28.  Mexico:  45
  29.  Spain:  43
  30.  Finland:  39
  31.  Italy:  38
  32.  South Korea:  38
  33.  Switzerland:  38
  34.  Singapore:  36
  35.  Mali:  35
  36.  Chile:  33
  37.  Turkey:  33
  38.  Argentina:  27
  39.  Netherlands:  25
  40.  Syria:  25

"Cuba" often appears as "Mark Cuban", Jordan is the country most often confounded with a major celebrity and/or IT/tech personality ("Michael Jordan" and "Jordan Hubard" respectively), "Mali" is most likely to be confused with a car or AMD chipset, the United States has multiple possible references (US, U.S., USA, U.S.A., America, American), some of which are confounded (e.g., "North America"). But this provides a rough sense.

Analysis is largely awk, grep, sort, uniq --- standard shell tool stuff. Amazingly powerful though really. Crawl via wget which performed wonderfully as usual. Thanks to HN/YC for not throttling too aggressively.