Is it practical to use Twitter for service logs?

Question

Recently my boss suggested that we post logs of our service that is hosted on Windows Azure into twitter. His idea is that it's conveniently readable and if Azure cloud fails logs persist. He also said, lots of people do this, I Googled and couldn't find any data about this.

Currenly my concerns are the following. First, perhaps twitter is not very convenient for logging something that can happen very frequently (for example, when our service role starts each instance writes tens of records over several minutes). Second, even if twitter EULA allows such use at this moment it might be changed in future to cut off tweets that occur too often to be human-originated.

Is it practical to post service logs on twitter? What to consider if we implement this?

Answer 1

No. Watch the Twitter engineering talks they openly admit that they can accept some failure in some case loss of messages and no-notifications without incident. Also I would be concerned about the account being auto-blocked for what looks like abuse (breaking a certain rate-limit trigger etc..) Not a good idea,

Although what you could do, is after you capture/process the messages in your own logging persistence service, you could filter some out to twitter as secondary (aka not really critical) store..

Answer 2

Personally I don't like being dependent on external services when it isn't absolutely necessary. As you say, if one day the EULA changes, APIs change, anonymous hacks twitter or twitter stops its services entirely, then you'll have to deal with it. It could essentially boil down to higher costs in support and in maintenance one day. Also, it's not easily extensible.

Things which can easily done by yourself shouldn't be outsourced. I'd rather stay in control over the whole process, especially if it's as simple as sending a notification.

I don't say you should reinvent the wheel, but I'm pretty sure it's easy to use something else, be it custom or premade.

Edit: I think the 140 characters limit of a twitter message severely limits it as a reliable carrier. You most likely have to dis- and reassemble messages that are longer than the limit. What if parallel services twitter via the same account and the messages don't arrive in order? You'll have to use a custom format just for twitter. I bet there're more technical limitations like that.

Answer 3

Sounds almost like a PHB's idea from Dilbert comics. The more I think about it, the funnier it gets. In short: No, unless you want to become laughing stock.

It won't work anyway, Twitter has strict anti-spam limits, it's meant for status updates from human and no one is likely to post 1000+ status updates per day :).

Answer 4

Use an external log server.

Twitter is neither secure, nor reliable.

You shouldn't post the logs on a publicly readable medium, since you never know if it contains sensible data or not.

Answer 5

I would not be in favour of this either. I don't see why you'd want to do it either.

It might be worth asking him who actually uses twitter for automated logging - I haven't come across it myself and as far as twitter is used as an update for service issues for some suppliers, it is run by a human in the company's service desk for the purposes of information to end user, not as a service logging objective. IF Azure Cloud fails, nothing's going to be logging anything anyway.

Answer 6

No. Use syslogd hosted via any cheap ISP or on your own BSD server exposed to the internet via your corporate firewall.

man logd:

NAME
 syslogd -- log systems messages

SYNOPSIS
 syslogd [-468ACcdknosuv] [-a allowed_peer] [-b bind_address]
     [-f config_file] [-l [mode:]path] [-m mark_interval]
     [-P pid_file] [-p log_socket]

DESCRIPTION
 The syslogd utility reads and logs messages to the system console, log
 files, other machines and/or users as specified by its configuration
 file.

Answer 7

Hmmm, I think this is for what Loggly was created.

Loggly is a cloud logging system based on Apache Solr technology. As they say, it can handle millions of events per second. Seeing your needs, this solution can fit. In addition, they exposes a search API using a sort of custom query language.

Answer 8

If twitter decide they dont like you using their service to keep your business logs (I wouldent blame them) then they are well within their rights to just shut your user account even if it dosent breach the EULA. Do you really want to keep all your logs on something so volatile? Its hardly expensive to impliment your own logging system.

Answer 9

I would only use it as an email substitute to distribute administrative or trouble-shooting messages. It is not suited for critical transactions.

Answer 10

At most I can see Twitter or similar services being useful to point notices of high priority incidents, things that in older applications would generate an automatic text message or beeper message to a sysadmin or dba (and probably still should).
Using it as a replacement for your application logs? Idiotic idea.

Answer 11

Or you can use an Open source implementation twitter like Status.net et make available only for admins.