There is currently a large disparity among business confidence survey results for Queensland. The Sensis and Westpac Group-CCIQ reports for the March quarter suggest business confidence is relatively low, but the NAB report for the month of April, which the Queensland Treasurer frequently cites, is much more positive, with Queensland leading Australia on business confidence (see the Brisbane Times article Queensland businesses among most pessimistic in nation). But, as I’ll explain below, although the NAB survey might give reliable estimates of business confidence at a national level, there is very likely a large margin of error for its Queensland estimates, and, in my view, the Westpac Group-CCIQ estimates are likely to be most accurate for Queensland. The sample sizes for the State-level estimates in the NAB and Sensis surveys are so small that the resulting State-level estimates of business confidence have very high margins of error.

Let’s first consider the key findings of the different surveys:

• According to NAB’s Monthly Business Survey, business confidence in Queensland was +7, which appears to mean that optimistic businesses outnumber pessimistic businesses by 7 percentage points (i.e. 53.5% of businesses optimistic 46.5% pessimistic), the highest level among Australia states;
• The Sensis Small Business Index Survey for March found Queensland had an index score for the net balance of business confidence of +10, the second lowest in Australia, beating only SA which was at +6, and much lower than the national index of +27; and
• The Westpac Group CCIQ Pulse Survey found business confidence in the Queensland economy was at its lowest level in six years in March quarter 2015 at an index level of 37.7, which a table at the back of the survey report suggests is a “poor” reading, and that 50 per cent of Queensland businesses expect the economy to get weaker, and only 14 per cent expect it to get stronger.

There are definitional differences for the indices, timing differences (March versus April 2015), and issues of coverage, with the Sensis and Westpac Group-CCIQ surveys focussing on small and medium enterprises, but my feeling is that the surveys cannot be reconciled according to these differences, given the NAB survey appears much more positive for Queensland than the others. For instance, I can’t think of any change in April that would have caused Queensland businesses to become more confident than they were in the previous month.

So I would look instead at the sample sizes of the different surveys. Sample size is very important because it has a very large impact on the reliability of survey estimates. The Westpac Group CCIQ Pulse survey has the largest sample size for Queensland businesses, typically 800+ each quarter. Sensis surveys only 183 Queensland businesses, while NAB reports surveying 410 businesses across Australia, which means it may have only surveyed around 80 to 100 Queensland businesses.

With such a relatively small number of respondents in Queensland, the margin of error associated with the NAB survey is likely to be larger than that for the Sensis survey and much, much larger than that for the Westpac Group CCIQ Pulse survey, which has the type of sample size you really need to have confidence in the results for Queensland. Consider how the margin of error (95% confidence interval) for a proportion varies with the survey size in the figure below (N.B. for the purpose of the figure I’ve assumed the true proportion in the population that the survey is trying to estimate is 50 per cent; also see the nice figure in the Wikipedia entry on margin of error).

Finally, despite all the attention given to business confidence measures, it is possible that survey measures of business confidence don’t give us much of a guide to the future, or at least they don’t tell us anything we don’t already know based on looking at the most recent economic data. That was the conclusion of a very interesting RBA discussion paper from 2001 What do sentiment surveys measure? I highly doubt the survey techniques would have improved enough since then to doubt the currency of the paper’s findings.