The independent component analysis (ICA) tractography method has improved the ability to isolate intravoxel crossing fibers; however, the accuracy of ICA is limited in cases with voxels in local clusters lacking sufficient numbers of fibers with the same orientations. To overcome this limitation, the ICA was combined with a ball–stick model (BSM) [“ICA+BSM”]. An ICA approach is applied to identify crossing fiber components in voxels of small cluster, which are maximally independent in orientation. The eigenvectors of these components are numerically optimized via the subsequent BSM procedure. Simulation studies for two or three crossing fibers demonstrate that ICA+BSM overcomes the limitation of the original ICA method by refining regional ICA solutions in diffusion measurement of a single voxel. It shows 2°–5° of angular errors to isolate two or three fibers, providing a better recovery of simulated fibers compared with ICA alone. Human studies show that ICA+BSM achieves high anatomical correspondence of corticospinal tracts compared with postmortem corticospinal histology, yielding 92.2% true positive detection including both lateral and medial projections, compared with 84.1% for ICA alone. This study demonstrates that the intravoxel crossing fiber problem in clinical diffusion MRI may be sorted out more efficiently by combining ICA with BSM. Magn Reson Med 70:441–453, 2013. © 2012 Wiley Periodicals, Inc.