Where did the vegetation go?
Citizen scientist oregano wrote in this morning with an observation:
And they weren't the first to notice a change. Yesterday, booklady wrote:
If you've been on the project for the last year or so, you've noticed that the current season of Biodiversity Detective has a ton of vegetation pictures in it AND is taking forever to finish. Part of the slow pace was because of the volume of images we uploaded, but that didn't seem to explain it all. Lead investigator Forest Isbell was digging into the classification data output a week or so ago and noticed that unfortunately and frustratingly our custom retirement rules didn't seem to be working on this particular season!
As you may recall from other blog posts and discussions on the talk boards, we set up our retirement rules in a way that we hoped would maximize our volunteers' time, effort and expertise. As such, we planned (and succeeded, in the first two seasons!) for the following:
With these rules in place, the easier-to-identify images should retire quickly and the trickier ones stick around so that we have high confidence in the ultimate classification. However, a bug somewhere in the Zooniverse processing side of things apparently meant that - even though our custom rules were listed as active - EVERYTHING was requiring 15 classifications to retire!! Well, that explains the slow pace, doesn't it? If each of the thousands of images of blowing vegetation require 15 independent classifications, it's bound to take a while to wrap up the season. And it's bound to be frustrating for our volunteers who are trying to find wildlife and feel like their time might be being wasted!
Luckily, the wonderful Sarah Huebner from Snapshot Serengeti was able to fix the script early this week so that our retirement rules worked as they were supposed to. Additionally, we've lowered our threshold to 10 classifications for the 'everything else' category. Suddenly, many of the images of plants, cars, people and deer that had been lingering waiting for the 14th or 15th classification to come through retired! What remains in the photostream are the animal-rich images that need a few more eyes. At the rate you all are working, we'll likely be done with the current Biodiversity Detective season in a matter of days or weeks rather than months! So if you haven't been over to that workflow in a while, try it out - we think you'll be excited about what you find!
Moving forward, we expect to not only have our custom retirement rules working but to have many of the vehicle, people and plant-only images retire in the Animal or Not workflow, with only the captures of wildlife moving on to the classification section of Biodiversity Detective. We will also be rolling out some awesome new machine-learning components that will help us make the most efficient and effective use of our volunteers' contributions - stay tuned for a blog post about that in the next month or so! We anticipate that what you are experiencing now in Biodiversity Detective will be in the new normal, and we couldn't be happier.
What's happening? Just started today and have seen around 20 beautiful images in a row with interesting animals - none of the usual vegetation pics to wade through and many in new settings. All dated 2018. Is this an old data set?
And they weren't the first to notice a change. Yesterday, booklady wrote:
Quick question - every picture(s) I have done today had animal(s) in it [so I can't stop]. Is this because we are getting to the end of 2018? It has been very exciting!
If you've been on the project for the last year or so, you've noticed that the current season of Biodiversity Detective has a ton of vegetation pictures in it AND is taking forever to finish. Part of the slow pace was because of the volume of images we uploaded, but that didn't seem to explain it all. Lead investigator Forest Isbell was digging into the classification data output a week or so ago and noticed that unfortunately and frustratingly our custom retirement rules didn't seem to be working on this particular season!
As you may recall from other blog posts and discussions on the talk boards, we set up our retirement rules in a way that we hoped would maximize our volunteers' time, effort and expertise. As such, we planned (and succeeded, in the first two seasons!) for the following:
- Humans/vehicles: Retire if first two classifiers select 'human or vehicle' or majority of five agree
- Images of vegetation: Retire if first three classifiers select 'nothing there' or majority of five agree
- Deer: Retire if first three classifiers select 'nothing there' or majority of five agree
- Everything else: Retire if equal to or more than ten classifications are submitted
With these rules in place, the easier-to-identify images should retire quickly and the trickier ones stick around so that we have high confidence in the ultimate classification. However, a bug somewhere in the Zooniverse processing side of things apparently meant that - even though our custom rules were listed as active - EVERYTHING was requiring 15 classifications to retire!! Well, that explains the slow pace, doesn't it? If each of the thousands of images of blowing vegetation require 15 independent classifications, it's bound to take a while to wrap up the season. And it's bound to be frustrating for our volunteers who are trying to find wildlife and feel like their time might be being wasted!
Luckily, the wonderful Sarah Huebner from Snapshot Serengeti was able to fix the script early this week so that our retirement rules worked as they were supposed to. Additionally, we've lowered our threshold to 10 classifications for the 'everything else' category. Suddenly, many of the images of plants, cars, people and deer that had been lingering waiting for the 14th or 15th classification to come through retired! What remains in the photostream are the animal-rich images that need a few more eyes. At the rate you all are working, we'll likely be done with the current Biodiversity Detective season in a matter of days or weeks rather than months! So if you haven't been over to that workflow in a while, try it out - we think you'll be excited about what you find!
Moving forward, we expect to not only have our custom retirement rules working but to have many of the vehicle, people and plant-only images retire in the Animal or Not workflow, with only the captures of wildlife moving on to the classification section of Biodiversity Detective. We will also be rolling out some awesome new machine-learning components that will help us make the most efficient and effective use of our volunteers' contributions - stay tuned for a blog post about that in the next month or so! We anticipate that what you are experiencing now in Biodiversity Detective will be in the new normal, and we couldn't be happier.
Comments
Post a Comment