Study links deep-learning instability to fractal basins

A study by University of Sydney researchers proposes that intricate, fractal-like “riddled basins” in neural-network training may help explain why tiny changes to a model’s starting parameters can lead to different outcomes. The university says the effect persisted in deterministic computing environments, where repeated runs otherwise produce identical results. ^[2]

The university’s account reports that the researchers analyzed about 500,000 networks across tasks including image recognition and text processing, finding signatures consistent with riddled basins across architectures, datasets and training methods. In one image-recognition network, changing a starting setting by one bit led to a different internal configuration in about 48 percent of cases. ^[2]

The findings offer a proposed mathematical explanation for limits on reproducibility; they do not show that every neural network behaves this way or establish a practical fix. The university says lowering the learning rate improved reproducibility in the study but reduced learning effectiveness. ^[2]