This is a guest post by Ilan Rabinovitch, Director of Product Management at Datadog. The convergence of rapid feature development, automation, continuous delivery, and the shifting...by Ilan Rabinovitch
August 24, 2017
Don’t let the hardboiled-sounding name of our latest integration scare you off, because this monitoring service is a great way to get notified when one of your mission-critical scheduled tasks suddenly sleeps with the fishes. Dead Man’s Snitch is an uptime-monitor for cron or periodic jobs like backups or batch processing, and it alerts you when your jobs don’t run so you can investigate before it becomes a problem.
Using Dead Man’s Snitch is dead easy: it only takes a few seconds to set up a job-specific alert, a.k.a. a “Snitch,” and then you have your job curl that Snitch’s unique endpoint every time it runs.
$ rake pgbackups:archive && curl https://nosnch.in/c2354d53d2
You can also pass optional messages which is good for reporting how long your job takes to run:
$ m=`time ( your_command &> /dev/null) 2>&1` && curl -d "m=$m" https://nosnch.in/c2354d53d2
If your Snitch doesn’t check in within the time period you set, Dead Man’s Snitch will send you an alert via email, push notification, or most recently, as a PagerDuty Incident so your on-call team members will know immediately if your scheduled jobs go missing.
When your job starts reporting again, Dead Man’s Snitch will resolve the incident letting you know that everything is running smoothly.
“Quiet monitoring is great, but silent monitoring is terrifying. Dead Man’s Snitch gives us confidence that our monitoring systems are quiet for the right reasons.”
— Harry Marr, CTO, GoCardless.com
Integrating your Dead Man’s Snitch and PagerDuty accounts is easy and should take less than a minute.
3. You’ll be taken to pagerduty.com where you’ll authorize Dead Man’s Snitch by entering your PagerDuty email and password.
4. Lastly, create a new service for Dead Man’s Snitch and you’re good to go.
5. Dead Man’s Snitch will now create and manage incidents when your snitches go missing.
Dead Man’s Snitch offers an easy-to-use REST API and support for email check-ins, giving you multiple ways to keep an eye on your Snitches from any platform. Free iPhone and Android apps are also available and the service’s brand new Slack integration means that you can get updates on the go and right in your team’s Slack channel.
Dead Man’s Snitch is a product of Collective Idea, a software development consultancy based in Holland, Michigan.
Create your Dead Man’s Snitch account today and start kissing silent failures goodbye.