To address the needs of modern, data hungry machine learning algorithms, audio researchers often turn to crowdsourcing with the aim of hastening and scaling their efforts for audio annotation and audio quality evaluation. We research best practices for performing crowdsourced audio annotation and quality evaluation with the aim of increasing annotation quality, throughput, and user engagement.