The impact of rapid development of coastal aquaculture on aquatic environments is an important topic in environmental science. Quantitative assessment of the impact of aquaculture on sediment heavy metal pollution has been challenging because of the complex conservative-nonconservative behavior of heavy metals in coastal brackish waters. In this study, Sansha Bay, Fujian Province, the world’s largest yellow croaker cage culture area, was used as a research area for offshore aquaculture. Using aquaculture data recorded by remote sensing images combined with the relationships between sedimentary heavy metals and salinity, this study sought to analyze the effects of aquaculture on sediment heavy metal pollution. The results showed that over the past 15 years, the area of cage culture in Sansha bay has increased from 9.1 km2 to 33.4 km2, and the maximum intensity of cage culture per square kilometer has increased from 3% to 22%. As a result, the average values of Cu, Zn, Cd, and Pb levels in the cuprophilic elements in the culture area increased by 44%, 11%, 15%, and 17%, respectively, compared to non-farmed areas, and the slope of the conservative regression line with an increase in salinity decreased by 27%, 35%, 18%, and 2%, respectively. The average values of the siderophile elements Cr, Mn, and Ni in the breeding area increased by 16%, 15%, and 29%, respectively, compared to those in non-farmed areas. The results of potential ecological risk evaluation showed that Cd is a potential environmental pollutant in the surface sediments of Sansha Bay, and Sansha Bay as a whole is at a medium ecological risk level.