Weeds are among the major threats to cotton production. Overreliance on herbicides for weed control has accelerated the evolution of herbicide-resistance in weeds and caused increasing concerns about environments, food safety and human health. Machine vision systems for automated/robotic weeding have received growing interest towards the realization of integrated, sustainable weed management. However, in the presence of unstructured field environments and significant biological variability of weeds, it remains a serious challenge to develop robust in-crop weed identification and detection systems. To address this challenge requires the development of annotated, large-scale image datasets of weeds specific to cotton production and date-driven machine learning models for weed detection. Among various deep learning architectures, a diversity of YOLO (You Only Look Once) detectors is well-suited for real-time application and has enjoyed great popularity for generic object detection. This study presents a new dataset (CottoWeedDet12) of weeds that are important to cotton production in the southern United States; it consists of 5648 images of 12 weed classes with a total of 9370 bounding box annotations, collected under natural light conditions and at varied weed growth stages in cotton fields. A novel, comprehensive benchmark of 25 state-of-the-art YOLO object detectors of seven versions including YOLOv3, YOLOv4, Scaled-YOLOv4, YOLOR and YOLOv5, YOLOv6 and YOLOv7, has been established for weed detection on the dataset. Based on the Monte-Caro cross validation with 5 replications, the detection accuracy in terms of mAP@50 ranged from 88.14% by YOLOv3-tiny to 95.22% by YOLOv4, and the accuracy in terms of mAP@50[0.5:0.95] ranged from 68.18% by YOLOv3-tiny to 89.72% by Scaled-YOLOv4. All the YOLO models especially YOLOv5n and YOLOv5s have shown great potential for real-time weed detection, and data augmentation could increase weed detection accuracy. Both the weed detection dataset and software program codes for model benchmarking in this study are publicly available, which are expected to be valuable resources for promoting future research on AI-empowered weed detection and control for cotton and potentially other crops.