Building an AI Twitter Bot to Guess Photo Locations

Learn how we built and deployed the @WhereML Twitter bot that can identify where in the world a picture was taken using only the pixels in the image. We'll dive deep on artificial intelligence and with the MXNet framework and also talk about working with the Twitter Account Activity API.

Machine Learning in Production with Twitter Bots

In this session attendees will learn how we designed, built, and deployed the @WhereML Twitter bot which can identify where in the world a picture was taken using only the pixels in the image. We'll dive deep on artificial intelligence and deep learning with the MXNet framework and explore the Twitter Account Activity API.

Who Should Attend This Session?

You should attend this session if you are interested in learning more about modern machine learning tools and processes. You should also attend this session if you're interested in production deployments of machine learning models.

Architecture

The WhereML twitter bot is built on the LocationNet model which is trained with the Berkley Multimedia Commons public dataset of 33.9 million geotagged images from Flickr and other creative commons sources. The model is based on a ResNet-101 architecture and adds a classification layer that splits the earth into ~15000 cells created with Google's S2 spherical geometry library (the only part of the talk that's not in Python). This model is based on prior work at Berkley and Google.

In this session we'll start by describing AI in general terms for audiences who are unfamiliar with how it works. Then we'll dive into deep learning and the Apache MXNet machine learning framework, numpy, and scikit-learn. We'll describe the LocationNet model in detail and show how it is trained and created and areas for improvement. Finally, we'll talk about the Twitter Account Activity Webhooks API and how to interact with it using an API Gateway and AWS Lambda function.

Attendees are encouraged to interact with the bot in real-time on twitter at @WhereML.

Key Takeaways:
- The latest and greatest framework and models don't matter if you don't understand your core questions
- An architectural change can have vastly more powerful effects than just throwing more data at a problem
- Bigger models are not always better
- Machine learning is fun and you can perform experiments rapidly
- Twitter's account activity API is brilliantly designed but a little haphazardly implemented

All code used in this project is open source, built live on twitch.tv/aws, and attendees are encouraged to experiment with it.

Comments

Comments are closed.