3
$\begingroup$

I am using this index page to download GDR2 data files.

http://cdn.gea.esac.esa.int/Gaia/gdr2/gaia_source/csv/

But it keeps giving a different number of links. Sometimes it shows 16,000 files, sometimes 31,000 and the max I have seen is 61,237. What is the correct number I should download? GAIA GDR2 documentation does not list the total number of files available.

Interestingly, if the index lists less than 60k files, it still has the closing /html tag so it would appear I have a complete index file, but the contents are not complete.

$\endgroup$
  • 1
    $\begingroup$ Using curl, I got 61240 lines total, but that includes some headers, so I think 61237 is correct and your browser may just be timing out. In particular, the last file on my list is "GaiaSource_999922404314639104_1000172126596665472.csv.gz", which seems correct, since the first number increases with each file and seems to finally "wrap around" with the last file. $\endgroup$ – barrycarter Apr 29 '18 at 3:54
  • $\begingroup$ Thanks, I did more tests and 61237 seems to be the correct number. Add this as an answer and i will accept it. $\endgroup$ – Ags1 Apr 30 '18 at 10:04
  • $\begingroup$ Thanks. I'm not really into the points thing, but feel free to post it as your answer. $\endgroup$ – barrycarter Apr 30 '18 at 17:04
0
$\begingroup$

barrycarter provided the answer:

Using curl, I got 61240 lines total, but that includes some headers, so I think 61237 is correct and your browser may just be timing out. In particular, the last file on my list is "GaiaSource_999922404314639104_1000172126596665472.csv.gz", which seems correct, since the first number increases with each file and seems to finally "wrap around" with the last file. – barrycarter Apr 29 at 3:54

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.