Infield pre-sorting is intended to remove processing (inferior) fruit that are not suitable for the fresh market, so that growers could handle sorted apples differentially in postharvest storage and packing to achieve cost savings. To achieve this goal, we developed an apple infield grading and sorting system, in which fruit singulation, rotation, and transportation were achieved by using simple, compact pitch-variable screw conveyors, fruit grading (size and color) was accomplished with a low-cost imaging system, and fruit sorting was done by using paddle sorters. Experiments were conducted for ‘Red Delicious’ and ‘Golden Delicious’ apples to evaluate the overall performance of the infield grading and sorting system in terms of grading repeatability (i.e., chances of each apple that would be graded into the same quality grade in multiple runs and different lanes), bruising damage, and sorting accuracy (consistency between the imaging-based grading results and destinations). Results showed that the grading repeatability rates of the system were above 90% and 81% for intra- and interlane grading, respectively. The system achieved above 99% sorting accuracy for the system throughputs of 7.5, 9.0, and 10.5 fruit s-1, while 100% of sorted apples were graded Extra Fancy and 55% or higher of the apples incurred no bruising damage at all after grading and sorting. The developed infield grading and sorting system is compact and robust in performance, and it can meet commercial infield sorting needs.