This year while brainstorming what I should talk about in my BrightonSEO I came up with the marvellous idea of creating a twitter bot to check hreflang errors. Some may say marvellous, others, “nerdy procrastination”.
I spent a couple days with the Tweepy API and built the crude hreflang checker class to pass the links that were tweeted by the keen beans in the audience.
Firstly I tested locally, sending around 150 tweets until the bot was consistently working, next up was getting in onto a server.
I have a $10 linode box i use for lightweight tasks and it was just the workhorse I needed to do this task.
Finally I had people in the office tweet Dave to ensure everything was running smoothly. Everything seemed great, I was pretty excited to see the results.
Where I went wrong
You can find the code in my GitHub under twitter reply bot.
Firstly a disclaimer – the code isn’t beautiful but in the short period of time I had did the trick.
I was using the streaming function of the Tweepy library.
As of publishing this is the most up to date version of the library http://docs.tweepy.org/en/v3.5.0/streaming_how_to.html
if __name__ == '__main__':
stream_listener = StreamListener()
stream = tweepy.Stream(auth=api.auth, listener=stream_listener)
stream.filter(track=["@daveseobot"])
This is the piece of code used to initialise the stream functionality all works well here.
class StreamListener(tweepy.StreamListener):
def on_status(self, status):
tweet = status._json
print(tweet)
q.put(tweet)
grab_data_from_queue()
def on_error(self, status_code):
if status_code == 420:
return False
This is the StreamListener class we call in the main function. Essentially the tweet is converted to json, put into a queue and then processed with the queuing function.
Have you seen the error yet?
I print() every tweet to the console. I did this during dev and sloppily left it in there when I put it live.
The major issue with this is that it doesn’t handle UTF characters and will throw an error.
And guess what that’s an issue because, some people have UTF-8 characters in their name.
The bot broke instantly while it was processing the tweets, leaving me rather embarrassed as I realised something had gone terribly wrong, due to the fact the phone in my pocket wasn’t vibrating at all!
Core takeaway
Always ensure you get rid of unnecessary print statements and always handle UTF-8 encoding.
Better luck next time eh?