To accurately characterize the effects of dissolved oxygen （DO） concentration on koi behaviors， two novel quantitative indicators， Planar Motion Direction Entropy （PMDE） and Group Aggregation Index （GAI）， based on object tracking are proposed. First， a modified H-SORT object tracker was constructed by introducing the HOG contour feature into the original SORT algorithm. Then， the matching mechanism was optimized via a weighted fusion strategy to enhance the stability of real-time koi tracking. Finally， the behavioral states of koi under different DO concentrations were compared and analyzed based on the two quantitative indicators （PMDE and GAI）. The results showed that the H-SORT algorithm increased the 10-second tracking retention rate of koi by 27.8%， which effectively improved the recognition accuracy in cases of fish school occlusion. The Spearman correlation coefficient between PMDE and DO concentration reached -0.966 9， an increase of 0.240 7 compared with the average swimming speed （Avg_Speed）， indicating that PMDE can effectively capture the behavioral characteristics of koi under different DO concentrations and reflect the impacts of dynamic DO changes and fluctuations on the swimming behaviors of koi. GAI exhibited a strong negative correlation with DO concentration with a coefficient of -0.924 0， an increase of 0.202 8 compared with the average distance （AD）， and the data fluctuation of GAI was within ±0.05. The improved tracking algorithm and the proposed quantitative indicators in this study can effectively quantify the effects of DO concentration on fish school behaviors， which provides a theoretical and technical basis for water quality monitoring and early warning based on fish school behaviors in aquaculture.