SLOOP: Symbiotic Learning Observatories for Organismic Photo-identification

Research supported in part by AFOSR DDDAS Program, Lincoln Laboratory, MISTI, NSF, and NUWC

The History of Sloop

Timeline

WACV98, SIGIR97, ECCV96: Use of local features in Animal retrieval examples led to ideas!

PhD 2002–Image Retrieval.  Search for sustainability-related application of technology commences.  Lloyd Gamble introduces salamander problem.

ACCV 2004–Multiscale local features, marbled salamander.

  • Few clicks and fast matching.
  • Retrieval paradigm for Individual Animal Identification and Animal Biometrics.
  • Medial axes and symmetry detection
  • First approach using generic visua features. 

CVPR 2004 — Affine invariant local features, marbled salamander.

  • May be too invariant, not selective enough.

JAE 2008 — Multiscale PCA marbled salamander.

  • Impact in long-term study.
  • Randomized representations.

VAIB 08 — Sloop v1.0 is released!

  • Relevance feedback.
  • Deformation Invariance.
  • Multiple preprocessing algorithms.

ICCV 09 — Scale-Cascaded Alignment, MS-PCA, SIFT salamander, Sloop v2.0.

  • SCA, many applications.
  • Exemplar based specularity removal.
  • Randomized graphs and manifold distances.
  • Aggregation.

2010 — Sloop Amop released. Also, Fowler’s Toad and Tiger Salamander prototypes.

2011 — Sloop Skinks

  • Citizen Science.
  • Crowd-sourcing relevance feedback.
  • Social media Sloop.
  • Assisted and automated modes. 

2012 — Sloop v2.7 operational for conservation. First pattern retrieval engine in use.

MCPR 2013 — Sloop v2.7 Gecko, Whale Shark

  • Hybrid contexts

MCPR 2014 — Large-scale Relevance Feedback, Special Session on Animal Biometrics.

Pattern Recognition 2015: Summarizes many results and adds new species. 

  • Speaker, Third Wildlife Photo-ID workshop, Findland
  • Boston University Image and Vision Computing Seminar

2017: First Vision-based Wildlife Management Workshop, ICCV 17 as co-organizer

2018: First Computer Vision for Animal Biometrics special issue (IET), co-editor.

WACV 2020 (workshop): Sloop Deep Learning using Appearance and Geometry 

Goals and Value Proposition

  1. Focus on Animal Biometrics for conservation of rare and endangered species. Typically small species.
  2. Several firsts for Individual Animal Identification:
    1. First application of generic visual features.
    2. First search engine model for animal ID.
    3. First Relevance feedback animal ID system (human-in-the-loop).
    4. First relevance feedback by crowdsourcing.
    5. First Vision-based Wildlife Management workshop with ICCV17
    6. First special issue on Animal Biometrics, IET 2018 
  3. Other elements of approach
    1. Feedback symbiosis: matching to ease the citizen scientist’s work, relevance feedback to improve algorithms. This is much different than simply using crowds to gather images.
    2. Interactive preprocessing for  segmentation, rectification and illumination.
    3. Novel methods.
      1. Scale Cascaded Alignment.
      2. Exemplar-based specularity.
      3. Hybrid Contexts for Recognition.
      4. Information-theoretic Relevance feedback.
      5. Information theoretic aggregation.
    4. Distributed system, short “idea to realization” cycle.
  4. Sloop consists of two servers: IPE (Image Processing Engine) and DEI (Data Exchange and Interaction).
    • Version 3.0: It sits on a Glassfish Server with Postgres bindings and makes use of DataNucleus/JPOX for DEI and uses Matlab/Octave for IPE.
      1. SloopLite is a desktop application for small collections.
      2. SloopMachine is a complete virtual machine.
    • Version 4.0: A RESTful system based on Postgres, and Node and Angular in DEI with matlab/octabe IPE.  Runs on all formats — web, handheld, phone.
    • Version 5.0: Cloud Migration
    • Version 6.0: Embedded and Cloud Application