I am using this index page to download GDR2 data files.
But it keeps giving a different number of links. Sometimes it shows 16,000 files, sometimes 31,000 and the max I have seen is 61,237. What is the correct number I should download? GAIA GDR2 documentation does not list the total number of files available.
Interestingly, if the index lists less than 60k files, it still has the closing /html tag so it would appear I have a complete index file, but the contents are not complete.
curl, I got 61240 lines total, but that includes some headers, so I think 61237 is correct and your browser may just be timing out. In particular, the last file on my list is "GaiaSource_999922404314639104_1000172126596665472.csv.gz", which seems correct, since the first number increases with each file and seems to finally "wrap around" with the last file. $\endgroup$