The deployment of smart city technologies offers local governments a chance to enhance citizens’ quality of life by tackling issues such as traffic congestion. Traffic management systems often utilise video-based traffic sensors (VBTSs) to analyse traffic patterns via camera images, which might capture sensitive data like location and driving behaviours. This capability can evoke perceptions of urban surveillance, raising privacy concerns and resistance, potentially hindering smart city initiatives’ success. Understanding factors influencing the social acceptance of these technologies is crucial for their successful implementation. This study applies privacy calculus theory and examines how citizens’ trust in government implementation of VBTSs and privacy concerns affect their social acceptance, using a stratified sample of 1920 US residents. Findings indicate privacy concerns negatively impact VBTSs acceptance at both general and local levels, while trust in government boosts general acceptance but does not affect local acceptance. Thus, our findings suggest privacy and trust play vital roles in determining the success or failure of a potentially controversial smart city initiative. These insights are valuable for governments at a practical level and for researchers at a theoretical level.