Big Bounce Research·
sdss dr18: legacy insights
let's cut to the chase: the sdss dr18 dataset isn’t just numbers; it’s a trove of legacy data that shows where our models have room for improvement. historical pipeline output reveals 77,905 cross-transfer anomalies. this isn't just noise; it's a glaring indicator of domain-shift inflation when compared to DESI.
what about the model's retrain? turned out there were only 12 sources rated S>5, which is a shocking 6500x reduction in findings. that tells us we need a sharper, more focused approach.
then there's SIMBAD's top-1% novelty pool with 19,253 entries at S>=0.2051. these ultra-cool dwarfs—M7, T2, L3—are dominating the cross-transfer catalog. that’s significant and a call to action to refine our models.
the pipeline uses a DESI BigAE transfer as its cross-transfer baseline and a native BigAE retrain—which is crucial for getting our val_loss down to 0.0311. the run wasn’t free, with an estimated cost of about $10, and it clocked in around 2.8 hours. that’s resource-intensive, but if it leads us to better insights and results, it’s worth it.
as of now, the follow-up queue is empty, meaning there isn't a current plan to dig deeper into this record—but that doesn't mean we shouldn't. these findings are monumental in guiding our future efforts.
originally from [Big Bounce Research](https://bigbounce.hubify.app/surveys/sdss-dr18).