Earlier this year, my brother asked me a simple question: “Hey, what do you think about the Arcade Fire?” See, as far as my family and close friends know, I’m still the same fervent music obsessive I was back when I was a teenager, the kid who spent most of high school lying on his bed with the new issue of Spin propped against his knees, lost in his big headphones. If anyone could be trusted with a music recommendation, it was me. But high school was a long time ago. You know how it goes: Now college-degreed and gainfully employed, I’ve simply lacked the time, strength, cash, and patience to keep up with the mutating musical tastes of young, hip America. I love music as much as ever, but—with my yearly CD and concert intake down roughly fivefold from my fanboy peak—I’ve become the type of music fan I’d once ridiculed: a casual music fan. If my seventeen-year-old self could see me now, he’d be horrified.

Which brings me back to the Arcade Fire, and my brother’s simple question. “I’ve read a lot of good things about them,” I sputtered in response. “Haven’t actually heard their music yet, though.” This was a shameful admission. As any young, hip American could tell you, the Arcade Fire is one of the most important bands in indie rock today, their big-deal status cemented by a so-glowing-you-gotta-wear-shades review of their first album on the music webzine Pitchfork. In fact, nearly every big new indie band of the past couple years has, at some point, received the Pitchfork stamp of approval. I decided, therefore, that the best way to re-access the strange world of music obsessives was to let Pitchfork be my tour guide. My plan: to buy ten of Pitchfork’s highest-rated CDs from this past June, listen to them, and get a feel for what kinds of music the most influential indie tastemakers in the country are digging. And because every good plan has an ulterior motive, here was mine: to find a new album that I could fall in love with. It had been a while.

On the off chance you haven’t heard of Pitchfork, here’s the backstory: In 1996, teenage slacker Ryan Schreiber launched pitchforkmedia.com from his parents’ house in suburban Minneapolis, writing all the CD reviews himself. Gradually, unpaid freelancers came aboard to contribute copy. Interviews and features (mostly top-100 lists) followed, but the meat of the site was, and remains, the reviews, which grade records on a ten-point scale. In 1999, Schreiber bravely threw his whole wad into running the still-insignificant site. He corralled advertisers and started paying his writers. Pitchfork grew in size and influence, soon becoming notorious for the “Pitchfork Effect”—a rave from Pitchfork can thrust an unknown band onto center stage at the Hammerstein; a bad review means “Nice knowin’ ya!” Today, Pitchfork employs half a dozen full-time staffers at its Chicago HQ. It’s far and away the most successful online music magazine on the scene.

Pitchfork now averages 160,000 visitors each day. It’s a safe bet, however, that of those 160,000, zero would admit to actually liking Pitchfork. Non-casual music fans love to complain about the site’s overarching editorial tone (“snarky,” “hipper-than-thou”: two overused terms that nevertheless apply here), as well as the CDs it chooses or declines to review (don’t trust any artist over thirty; marginal indie rock trumps marginal indie rap every time).

And then there’s the actual writing. Conventional wisdom states that Pitchfork prose is, um, casually edited, as well as pretentious to the point of impenetrability. But it’s fairly evident that Pitchfork is barely edited at all; vague language and wasted words abound. However, the weird, form-shattering reviews that Pitchfork was once known for (check out the review of Metallica’s St. Anger, for instance, or Autechre’s Untitled) seem to have disappeared from the site completely. The bulk of Pitchfork’s content is now meat-and-potatoes rock-crit, competent hackwork through and through. There’s nothing wrong with that, of course, but one misses the oddball experiments, even if they were occasionally self-indulgent. Nowadays, if Pitchfork’s reviews falter—which they often do—it’s in two crucial areas: critical insight (in lieu of substantive analysis, reviews offer superficial blandishments about “strong” choruses, “engrossing” melodies, and “affecting” vocals); and even worse, entertainment value, which you’d better believe counts for a lot in rock writing.

But as a friend of mine says, you read Pitchfork for the numbers at the top of the screen, not the words in the middle. These are, after all, reviews, not essays. Their goal—and it’s a noble goal—is to tell you which records most deserve your hard-earned iTunes credits. Though their reviewers tend toward mediocrity in their collective writing ability, historically, they’ve displayed solid (if narrow) taste in music. Broken Social Scene, Clap Your Hands Say Yeah, Band of Horses: all good-to-great bands, all recipients of Pitchfork raves.

In June, Pitchfork tagged fourteen albums with the label of “Recommended” or “Best New Music.” From that group, I selected ten at random and bought them. Actually, that’s a lie: I must confess a slight selection bias. White Stripes, Dizzee Rascal, and Pharoahe Monch were the only three Pitchfork-approved artists that I had heard of before, so I made sure to pick up their new CDs. As it turned out, Dizzee’s Maths + English and the Stripes’ Icky Thump were my two favorite albums of the month—the former a wry, smirking party record bolstered by some ingenious samples and the hot-fun-in-the-summertime anthem “Da Feelin’,” the latter a giddy tour of a rock ’n’ roll rag-and-bone shop in which Jack White flaunts his acting skills sometimes and his blues riffs always. Pharoahe’s Desire is also pretty darned good. My experiment was off to a fine start.

Next, I gave two dance records a spin: † by Justice, and Attack Decay Sustain Release by Simian Mobile Disco (the latter set me back twenty-one bucks—goddamn imports). Justice’s album is uneven, but at its best it’s goofy, gimmicky fun, earning its 8.4 rating. I ran into problems with Simian Mobile Disco, however. As former Village Voice critic Robert Christgau has pointed out, the esoterica-obsessed Pitchfork reviewers often seem so starved for hooks and beats they’ll heap praise on anything that emits even the faintest pop pulse. Thus, Pitchfork’s Jess Harvell writes that Attack’s “over-the-top friendliness, immediacy, and excitement has a cramped, amped-up hysteria, like all great bubblegum.” Now, as a connoisseur of great bubblegum, I must call bullshit here. The album is eager to please, sure, but silly playground-chant samples and bouncy rhythms alone do not elevate music to bubblegum status. Bubblegum is about songs: melodies and choruses and that added je ne sais gabba gabba hey. Attack’s linear synth workouts are in no way “bubblegum,” period.

Still, Harvell’s adjective-drunk review was well-written. He knows what he’s doing behind the home row, one of about three or four Pitchfork writers whose bylines are actually worth Googling. It’s always tempting to use “Pitchfork” as a synecdochic catch-all for all the site’s scribes (“Did you read what Pitchfork wrote about Liz Phair?”), but there’s a wide spectrum of writing abilities exhibited along the fringes of Pitchfork’s stable of forty-three contributors. There’s one thing that isn’t on display, though: vaginas! An unfathomable four out of the forty-three are chicks, lending the site a dramatically boy-centric skew. Consider albums six and seven in my experiment: Pig Destroyer’s Phantom Limb (grindcore) and Pissed Jeans’ Hope for Men (sludgecore). Both are extreme, fringe-y, and therefore masculine, and Harvell’s reviews of both are very boy. I couldn’t parse either of ’em, which reflects poorly on my manhood, I’m sure.

Fort Nightly, by the NYC-based White Rabbits, was album number eight. Of all the albums I listened to in June, Fort Nightly earned the clunkiest Pitchfork review. Joe Tangari is right to note the band’s “rhythmic intensity”—it’s the most striking aspect of their sound—but elsewhere, he hears things that simply aren’t there. (“A major second-wave ska influence”? There are some upstrokes, sure. But “frequently surf-inspired guitars”? Where?) I drew a blank on album number nine, Matthew Dear’s Asa Breed. To my ears, it’s partly pretty sonic wallpaper. I still can’t get a bead on it, and Pitchfork’s book report of a review isn’t terribly insightful. Finally, Wild Mountain Nation by Blitzen Trapper: It looks like a Pavement record, it sounds like a Pavement record, it’s actually pretty good. (It also sounds kind of like an Olivia Tremor Control album, minus the lo-fi psychedelic fiddle-dee-dee.) Pitchfork’s review of the album is fine.

“Fine”—it’s the great mean of Pitchfork editorial content. Schreiber would be nuts to cough up any more than the twenty-five bucks he reportedly pays writers per review. Then again, you’d be even crazier to base a month’s worth of music purchases solely on Pitchfork’s recommendations. And you’d have to be downright touched in the head to pay twenty-one bucks for the Simian Mobile Disco album. Trust me.