Conjuring Bullshit for Fun and Profit

Arthur C. Clarke — a British writer whose Wikipedia article I just skimmed — apparently once wrote: “Any sufficiently advanced technology is indistinguishable from magic.”

A nifty little quote, all things considered.

If you can remember the first time you booted a computer or tickled an iPhone’s multitouch screen or perhaps the first time you sent a message to a chatroom and a bloke several continents over instantaneously invited you to STFU: you probably know what Mr. Clarke is talking about.

The idea that the skirting of electrons across wafers of finely laced silicon yields the impossible complexity we enjoy on our computers, in our vehicles, and on our phones seems just as foreign as purple-tinged smoke unfurling from a wizard’s crystal sceptre.

Technology is magical — in the same way that any complexity seems magical until you’ve popped the hood and poked around a bit.

Indeed, as remarkable and esoteric as technology can certainly be, it is also decidedly not magical. Someone — who in all likelihood was a human being like you — had to roll up their sleeves and pull shit together so that you can watch cat videos on your cellphone at 30,000 feet.

While the overall result is admittedly magical (and I do believe that this sense of “magic” is very special and very real), the underlying innovation is also fiercely deliberate, incredibly complex, and often clever as a cartload of monkeys.

Broadly speaking, it seems that the magic card gets played in two very different scenarios:

The first typically involves a person who has been delighted by a nifty bit of technology and who doesn’t quite know how the darn thing works. For instance, that’s exactly how I felt the first time I encountered a glasses-free 3D display; that shit is unreal.

The second typically involves salespeople trying to move product at the expense of uninformed, non-technical consumers.

Shall we play mad-libs? Go ahead and strike the salespeople/consumers bit from the above and drop in any of the following pairs:

  • Entrepreneurs / Investors
  • Shitty developers / Managers
  • Writers / Readers
  • Agencies / Clients
  • Apps / Users

You’ll note that the sentence remains true.

Though most users are finally starting to work up a hint of tech literacy, the supernatural is often easier to embrace than the mundane: who wants to (or has the time to) study engineering when the techno trolls and code sorcerers are so much more compelling?

The point isn’t that more people should be engineers — though I do think it’s an endlessly fascinating field — it’s that there are loads of phony “startup” people out there exploiting the magic inherent to technology to further their own goals and line their own pockets.

They’re our industry’s equivalent of dishonest car mechanics — but with twice the bullshit and half the expert knowledge.

The net result of these shenanigans? Wasted capital, widespread deception, and the preponderance of grifters over actual innovators, inventors, and builders.

If you’re a technologist, I urge you to call bullshit early and often. Don’t let the hype machine run unchecked. Add your two cents.

If you’re a user, I urge you to get curious. Don’t assume that something is magical just because you can’t see the smoke and mirrors. Hire an independent contractor. Get a second opinion. Invest in innovation.

If you can’t find the value, chances are it doesn’t exist.

One Year Later: WTF?

Approximately 460.5 days ago, in a distant place (Florida) and a distant mood (happiness), I sat down to review my lot in life.

I wrote about the things that I thought made NY tech pretty neat. I shed a tear over the kindnesses of local “hackers” and even waxed poetic about the experience of being a founder in the heart of the Big Apple.

My delusion — whether contrived or genuine — was crystal clear. The words I wrote came from a place familiar to anyone who has ever felt like an outcast: I just wanted to fit in.


Reading these words now makes me feel like a complete sellout. How did I go from telling recruiters to get fucked to celebrating the warmth and kindness of the NYC startup phenomenon so quickly?

I outwardly celebrated what I inwardly mourned and boy do I feel like a cheap floozy now.

I’ve come to realize that if I don’t write about the stuff that pisses me off, I’ll be doing this community — my community! — a disservice. And that’s the last thing this increasingly hypersensitive, nepotistic and ferociously self-involved “innovation” community needs. More useful might be some Jergens and a box of kleenex.

We need to topple the bullshit artists from their soapboxes. We need to value knowledge and skill over pomp and persuasion. We need to ephasize learning and education over cockiness and swindlership — and no, Computer Science education isn’t a business plan.

Simply put, we need to innovate and not just revel in the smell of our own shit.

So let’s cut the crap. Personally, I’ll be tearing down my internal politician in hopes of reconnecting with the angry geek within. We cannot endure another Bravo reality show: the bubble is popping and the flies are congregating. Time to flick them away.

A Moment for Reflection

When I started this blog two years ago, I set out to capture a side of the tech world that I felt was being overlooked: the people behind the scenes — and the sense of community that binds them together.

"Your Startup Sucks", though silly, was imagined as a good guy’s travelog — a personal narrative probing the winding corridors of NYC’s business world. Its thesis was simple: dogs may eat dogs elsewhere but in NYC, we help each other out. That’s what makes us special. That’s what makes us important.

Over these past two years, I’ve been fortunate enough to meet thousands of people: founders, hackers, investors, writers, students, and leaders. Some among them sought success in business; others sought learning and fellowship. Most understood that the two weren’t so different.

Along the way, I’ve come to see starting up as being a bit like falling in love: passionate words whispered over drinks, late nights that turn into long weekends, fleeting moments of absolute (and terrifying) clarity… followed by the harrowing awareness that things will, in all likelihood, not work out in the long run. What then?

As it is with relationships, so it is with startups: the best and only balm for a broken heart is friendship and family. In this game of numbers, one cannot afford to wallow in “failure”; we must rely on one another to help us through the disappointments so that we can dust ourselves off and try again.

Economists are fond of saying that “a rising tide lifts all ships”; it is humbling to belong to a community that so brilliantly embodies this aphorism. We work together, we play together, and we learn together. We share in the good and the bad and we understand this: the journey is everything.

In this new year, I look forward to continuing our journey together — to see what we’ll build and what we’ll say, who we’ll meet and what we’ll do. Most of all, I look forward to sharing another year of ups, downs, ANDs, ORs, and NOTs with you.

Have a happy and healthy 2012!

We Need Someone to Bridge the Gap


It’s 48 hours before elections close for the NYTM Board. Last year I outlined my reasons for voting for Evan Korth of NYU/HackNY and here I’d like to explain the reasoning behind my vote for this year.

Let’s start by asking: What’s missing from the NYTM Board?

Take a look at the Board and you’ll see a cast of accomplished and inspiring people. Dawn Barber, Scott Heiferman, Esther Dyson, David Rose, etc.

It’s great to have them on the Board because they bring influence and credibility to the NYTM, which is now the largest meetup in the world and the center of gravity for the startup community in NYC.

The Board’s job is to steer the direction of the organization through empathy with its members’ needs, and also manage relationships with other important bodies (the city, stakeholders, etc).

Think about the former for a minute and you’ll realize the opportunity I’d like to get across. While the membership is composed of mostly first-time founders—the hackers & hustlers who are trying to figure everything out—the Board are far past that stage.

The NYTM has been built up to an incredible scale and influence, and now it’s time for it to execute on fulfilling the needs, hopes and dreams of its members.

Because of this I believe that we need someone to bridge the gap between the membership and the Board. And I think the best person to do that is Brandon Diamond.

If you’ve been to the NYTM, you know Brandon.

He’s the young nerdy-looking co-organizer of NYTM that also runs the Hacker Union, a close-knit meetup for NYC’s developer community. Brandon’s incredibly active in the tech community, and he’s a first-time founder himself.

As co-organizer, Brandon does the heavy lifting of throwing the monthly event (which he’s done for the past two years) without much say into the vision/direction of NYTM. Electing him to the Board will give him that say.

Since Brandon knows the Board members and NYTM intimately, not only can he empathize, but he can also translate.

One of my favorite quotes goes, “In the beginner’s mind there are many possibilities, in the expert’s there are few.” There’s so many directions the membership want to go with the NYTM that have not been addressed or given attention by the Board. Brandon makes a specific mention to address this in his campaign page because he knows better than anyone what’s going on.

As an example, consider the Student NYTM, a subset of the NYTM that connects like-minded students in the community and gives them free tickets to the monthly meetup. Brandon spent significant time brainstorming and helping the group launch because he recognized the importance of the student pipeline. What did the Board do? Well, they’re not involved.

Another example is Hack of the Month, which Brandon created to get more hackers involved in the meetup. This adds a ton of value to the NYTM and is something that only Brandon could pull-off.

Not only does Brandon have empathy for the community, but also he has vision and the dedication to execute it. He won’t just be an important member of the Board, he’ll become one of the most important leaders in our community.

For these reasons, I hope you will consider the importance of casting your vote for Brandon.

Go here to vote:

The opportunity to submit or modify your vote ends Tuesday, December 20th at 11:59pm.

Building MongoDB for Fun and Profit

This post was originally published on the College2Startup blog.

"How’d you land a job at 10gen?"

My name is Brandon Diamond, and I’m a Database Kernel Engineer at 10gen. I’m very active in the startup community having served as the producer of the NY Tech Meetup; I’m also involved in several other groups including the Hacker Union ( as well as the Brown University NYC Meetup.

The emphasis I’ve placed on community activism has helped me to build a great network of friends and colleagues; it has also afforded me the opportunity to learn a great deal about NYC startups and organizations. Over the course of the past year, I became friends with a number of engineers at 10gen; I even began to use MongoDB in a number of my own startup projects.

When I decided it was time to start a new job, I reached out to some of the friends I had made at 10gen and asked them about open positions. Between the engineering-oriented company culture, open source MongoDB code base, and 10gen’s strong ties to the startup world, I knew that 10gen would be a great fit for me. I came in for an interview and started the following week.

"So… what exactly do you do?"

The Database Kernel Team at 10gen focuses on the MongoDB database server and related systems. We construct features, implement improvements, and address bugs as well as other user feedback. We also help troubleshoot unusual behavior in users’ MongoDB installations.

Working on the Database Kernel Team has proven to be a fantastic learning experience. I spend a great deal of time working with low level aspects of a wide number of systems; the code itself is well architected and efficient. Importantly, there are a great many people who enjoy using MongoDB; it’s fun to work on an interesting, complex project that directly impacts so many diverse — yet technical — users.

"What’s it like to work at 10gen?"

10gen offers a uniquely awesome startup experience. The culture is technical, casual, and meritocratic. Between the fully stocked snack bar and the weekly office lunches, working at 10gen doesn’t quite feel like… well, work. The team is friendly, outgoing, and passionate — everyone is excited about what they’re doing and eager to share what they’re working on.

Hierarchically speaking, the company is largely “flat”; the CEO and CTO sit alongside the other engineers and both write a significant amount of code nearly every day. The emphasis is on progress, community, and great software engineering.

All in all, 10gen is a fantastic place to work if you’re a startup-minded engineer who doesn’t want to compromise on tech or miss out on the startup experience. I couldn’t be happier with my choice of employer.

Me, Myself, and the NY Tech Meetup


It Begins

Two years and four months ago, I got off the subway and walked through the doors of FIT for my first NY Tech Meetup. Inside, I expected to find startups and demos. Instead, I found a movement.

Later that night I sent Nate Westheimer the first of what would be many emails:

I love what you’re doing at the NYTM. Is there anything I can do to help?

Over the months, I’ve watched as the community has flourished and grown; I’ve worked hard to refine our meetup into an event worthy of the NYU Skirball stage and, more importantly, your time and attention.

I’ve trained and rehearsed hundreds of presenters and personally auditioned dozens upon dozens more. I’ve fought for openness, community, transparency, and action. I’ve rolled up my sleeves and gotten my hands just about as dirty as hands can get.

Two years and four months later, I’m still helping. Why? Because I love NY tech. That’s not campaign rhetoric: it’s my call to action, my modus operandi. It’s why I’ve volunteered for these two years, four months and counting.

My Platform (… in a nutshell)

I don’t run a business school, I haven’t grown a startup to 100 employees, and I certainly haven’t raised a billion dollars— but I have spent 2 years writing code, co-organizing the NYTM, and getting to know the community.

The NYC startup experience isn’t abstract to me: it’s what I’m doing right now.

The only skin I’ve got in the game is my own— I’m running for the NY Tech Meetup Board not as a representative of a huge startup or incubator program, but as your representative. I’ve given the community two years of sweat equity and dedication; now I’m asking for your vote so I can take things even further.

I stand for three things: community involvement, getting things done, and recruiting more hackers. It’s my intention to identify and empower community leaders as a means of fulfilling our potential as an organization. Meanwhile, we’ll collaborate more closely with our engineers, creatives, and builders to ensure that there’s a steady stream of talent to fuel our industry well into the future.

In Closing

I’ve been putting blood, bytes and tears into this group for a long time. It hasn’t always been easy, but it has always been rewarding. The NY Tech Meetup has incredible potential — potential that remains to be fully realized.

If you’ve attended a meetup or two, please take a look at my full platform at — a small resource I’ve put together for the election. I’d like to do what I can to help the NYTM grow into the community-powered grassroots organization it was born to be; but before I can do that, I need your help.

Why The MongoDB Hate?

Disclosure: I hack on MongoDB.
Update: Check out Eliot’s (10gen’s CTO) response here

I was a little surprised to see all of the MongoDB hate in this Hacker News thread (and later this similar proggit thread). I’m going to do my best to reply to these concerns directly and with a minimum of breast-beating.

There seems to be quite a bit of misinformation out there: lots of folks seem focused on the global R/W lock and how it must lead to lousy performance. In practice, the global R/W isn’t optimal — but it’s really not a big deal. Here’s why:

First, MongoDB is designed to be run on a machine with sufficient primary memory to hold the working set. In this case, writes finish extremely quickly and therefore lock contention is quite low. Optimizing for this data pattern is a fundamental design decision.

Second, long running operations (i.e., just before a pageout) cause the MongoDB kernel to yield. This prevents slow operations from screwing the pooch, so to speak. Not perfect, but smooths over many problematic cases.

Third, the MongoDB developer community is extremely passionate about the project. Fine-grained locking and concurrency are areas of active development. The allegation that features or patches are withheld from the broader community is total bunk; the team at 10gen is dedicated, community-focused, and honest. Take a look at the Google Group, JIRA, or disqus if you don’t believe me: “free” tickets and questions get resolved very, very quickly.

Other criticisms of MongoDB concerning in-place updates and durability are worth looking at a bit more closely. MongoDB is designed to scale very well for applications where a single master (and/or sharding) makes sense. Thus, the “idiomatic” way of achieving durability in MongoDB is through replication — journaling comes at a cost that can, in a properly replicated environment, be safely factored out. That’s another design decision.

Next, in-place updates allow for extremely fast writes provided a correctly designed schema and an aversion to document-growing updates (i.e., $push). If you meet these requirements— or select an appropriate padding factor— you’ll enjoy high performance without having to garbage collect old versions of data or store more cruft than you need. Again, this is a design decision.

Finally, it is worth stressing the convenience and flexibility of a schemaless document-oriented datastore. Migrations are greatly simplified and generic models (i.e., product or profile) no longer require a zillion joins. In many regards, working with a schemaless store is a lot like working with an interpreted language: you don’t have to mess with “compilation” and you enjoy a bit more flexibility (though you’ll need to be more careful at runtime). It’s worth noting that MongoDB provides support for dynamic querying of this schemaless data — you’re free to ask whatever you like, indices be damned. Many other schemaless stores do not provide this functionality.

Regardless of the above, if you’re looking to scale writes and can tolerate data conflicts (due to outages or network partitions), you might be better served by Cassandra, CouchDB, or another master-master/NoSQL/fill-in-the-blank datastore. It’s really up to the developer to select the right tool for the job and to use that tool the way it’s designed to be used.

At the end of the day, MongoDB is a neat piece of software that’s designed to be useful for a particular subset of applications. Does it always work perfectly? No. Is it the best for everything? Not at all. Do the developers care? You better believe they do.

Things haven’t always been peachy. That’s why we recommend all new users start with the 2.0.x series (frankly, it seems unfair to rail on a project when you’re two major versions behind). If you look back, you’ll find mistakes: but you’ll also find a team of dedicated people who have worked hard to fix those mistakes.

10gen has built a novel datastore that offers high availability, sharding, and schema-free design at a very specific cost. Bugs will be pushed, mistakes will be made, and systems will go down. There is no silver bullet.

If you’ve got a mission critical application and you’re looking for a datastore, the first question you should be asking isn’t about internals or anecdotes, it’s "how are the inevitable boo-boos handled?" If the answer isn’t “efficiently, transparently, and with a heaping spoonful of honesty” — like it is with 10gen — you’ve got bigger problems.

Something Worth Fighting For

I fear for the future of the PC. The whole of consumer tech seems to have embraced the idea of autopilot computing: we’ve traded our freedom of choice for hollow devices coated in black lacquer and chained to strict software repositories and restrictions. If we dare defy the manufacturer by running unapproved software, we risk remote deactivation and lawsuits.

I fear for the future of the Internet. As eagerly as we’ve surrendered general purpose computing, we’ve sold our right to access an unadulterated and open web. If we don’t take action to preserve net neutrality and to ensure that the carriers — and our representatives! — cannot strangle our Internet, soon we’ll find our web browsers just as crippled as our mobile devices.

I fear for the future of developers. With the workings and machinations of our computers buried beneath layers of abstraction, litigation, and restriction, future generations will be deprived of the discovery and learning that has characterized computing through the 80s, 90s, and 2000s. Those that do pursue programming will spend their time studying proprietary APIs and complex licenses with little hope of accessing source code let alone the hardware itself.

I submit that we’ve taken for granted the single most important aspect of the computing revolution: openness. The closed systems, networks, and technologies that loom on the horizon reek of 1984 and threaten to dissolve the progress humanity has made over these past decades. This is something worth fighting for.

Announcing 56k: A NY Tech News and Culture Podcast

Well, I’ve finally gone and done it. Meet 56k.

I’ve been wanting to put together a podcast for a long, dreadful time; something led me to believe that it’d be quicker to assemble than a high quality blog post. I was sorely mistaken.

If you listen (and I hope you do!), you’ll find something of a hitchhiker’s guide to the soft underbelly of the NYC tech world. It starts with an overview of “This Week in New York Tech” and ends with a sassy diatribe regarding MBA types’ eternal quest for tech co-founders. I plan on putting new episodes out every week.

So, what do you think? Drop me a line in the comments!