1
$\begingroup$

I am currently doing my PhD in computer science, and I have almost no astronomical knowledge. I am currently researching on duplicate detection in databases (so detection of multiple entries referring to the same real world object).

For that, I put my focus on datasets consisting mainly of numerical data (e.g. measurement data). However, there are hardly any data sets in the computer science domain. An idea from me was to take data sets from the astronomy domain. I have seen that there are, for example, star catalogs (such as Gaia GDR-3 or USNO-B1.0), but the interpretation of these catalogs is difficult for me.

Therefore, my question: Are there data sets in the field of astronomy that contain duplicates (for example, that the same star appears several times in the same database)? Or are there multiple datasets that contain/refer to the same star? If so, could you please point me to such datasets?

$\endgroup$
6
  • 1
    $\begingroup$ I'm not quite sure I understand. Are you asking for sources of badly-vetted data? The answer certainly is 'yes', and it will also be 'yes' to "are there errors in current databases. The question is: are they known where these errors are. Because... if they are known, they likely will be corrected... You might go through the revision history of e.g. the gaia catalogues, or also of others, e.g. the hubble guide star catalogue, compare revision 1.0 and 1.1 $\endgroup$ Commented Mar 29, 2023 at 17:45
  • 1
    $\begingroup$ Hi, exactly, I mean such dirty data. My goal is to develop a method, which is able to automatically detect (and remove) such duplicates (the records do not need to be exactly the same but similar). To develop and evaluate a method like this, I need a groundtruth. So records for which I know whether they are a duplicate or not. Do you know if something like this exist? Another possibility might be to use two data sources which contain the same stars, but then I need to know, which star in source A is the same as a specific star in source B. Thank you! $\endgroup$
    – Lasklu
    Commented Mar 29, 2023 at 21:04
  • 1
    $\begingroup$ Hi @ProfRob, but do I get the information in these sources which star in source A relates to which star in source B? Thanks for your help $\endgroup$
    – Lasklu
    Commented Mar 29, 2023 at 21:05
  • 1
    $\begingroup$ Asteroids. But detecting duplicate asteroids requires calculating their orbits. $\endgroup$
    – PM 2Ring
    Commented Mar 30, 2023 at 7:55
  • 1
    $\begingroup$ @uhoh Understood! Will keep a note :) [flag the comment once acknowledged] $\endgroup$ Commented Mar 30, 2023 at 15:12

1 Answer 1

3
$\begingroup$

Yes, astronomical objects are contained in multiple catalogues. These catalogues will be observations made at different times, with different instruments and at possibly different wavelengths.

Cross-matching the catalogues isn't foolproof and there is no universal recipe for cross-matching or universal identifier for astronomical objects. Real objects are brighter at some wavelengths than others; are variable; and they move!

Very often one matches the catalogues based on similarity of the catalogued objects' positions (RA and Dec). That requires knowledge of how precise the coordinates are and possibly how fast objects are moving on the sky (proper motions).

An excellent (but non-trivial to use/learn) resource is Topcat - a Java-based astronomical catalogue and display tool. It includes a handy interface to query and cross-match many different catalogues.

$\endgroup$
3
  • $\begingroup$ Thanks for your answer and hinting to cross-matching. I have searched a little bit on this topic and found that it is very similar to my use case. I am wondering if something like a groundtruth or a source of truth exist? So something for which cross-matching for a dataset (or a subset of it) has been checked (manually) to allow evaluation of newly developed cross-matching methods. $\endgroup$
    – Lasklu
    Commented Mar 30, 2023 at 10:43
  • $\begingroup$ That link is broken for me - perhaps because I'm based in Europe / The Netherlands? Otherwise it's rather unique for an answer younger than a week old ... $\endgroup$
    – Glorfindel
    Commented Apr 4, 2023 at 16:31
  • 1
    $\begingroup$ @Glorfindel their site appears to be down. The address is correct. $\endgroup$
    – ProfRob
    Commented Apr 4, 2023 at 16:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .