Stack Overflow just posted their annual developer survey for 2015 – http://stackoverflow.com/research/developer-survey-2015. I was more than a bit surprised that Salesforce development topped the list of most dreaded technologies. Now I’m not questioning their results, but you know what Mark Twain is misquoted as having said – there are lies, damned lies, and statistics, and this one didn’t quite pass the smell test for me. I know a lot of Salesforce developers at all levels, and if 73% of them dreaded working on the platform I would expect to be having quite a few conversations consisting of people complaining about the platform and how they are studying other technologies in the hope to escape those dreaded limit errors.
In the survey, dreaded is defined as “% of devs who are developing with the language or tech but have not expressed interest in continuing to do so.” I know many developers who have come to Apex from other languages. I can’t think of any of them who are looking to do something else. There are some great things about developing on the Salesforce platform, and some very annoying things as well. I can see how someone coming to Apex from another language might find aspects of it very frustrating (don’t we all?) but I think most of us find the advantages far outweigh the disadvantages. Enough so that we are generally very interested in continuing to work on the platform as it evolves.
Something isn’t quite right.
So I asked myself, who were the respondents to the survey? It must have been a fairly small sample of Salesforce developers, after all Salesforce doesn’t even appear on the list of popular technologies (defined as most used), so less than 7.8% of the respondents would be Salesforce devs. Of course, this could still be a statistically significant number, but it does suggest that Stack Overflow does not necessarily attract large numbers of Salesforce developers.
And why would it? Any experienced Salesforce developer is much more likely to be active on the similarly named and often confused Stack Exchange – specifically Salesforce.StackExchange.com. If you look for answers on Salesforce or Apex questions, you’re much more likely to be directed there than Stack Overflow. Looking at Stack Overflow, the tag Apex has 667 questions. Apex on Stack Exchange has 7000 questions. Does this mean that Salesforce developers who are happier on the platform are more likely to be on Stack Exchange than Stack Overflow? Are the Salesforce devs on Stack Overflow more likely to be part time on Salesforce where those on Stack Exchange are full time on the platform? Are developers who spent their time on Stack Exchange less likely to have seen the ads for the survey (which are noted as appearing on Stack Overflow sites)? I have no idea. But I’d bet there’s a selection bias at play here, and I’d bet it’s significant.
So I call bullshit on this statistic. I think it falls into the category Stack Overflow uses to describe the results where “These results are not unbiased. Like the results of any survey, they are skewed by selection bias, language bias, and probably a few other biases.”
I think there is some extreme selection bias going on here. Nothing against Stack Overflow by the way – they are very transparent about the results and potential for bias. Unfortunately, I’m afraid a lot of people will look at that number and assume it means something, where in fact the hitherto unasked question – of how many Salesforce developers actually dread the technology they are working on, remains unanswered.