User Agents are just "free-text" HTTP Header fields; there is absolutely no standard format for them and any software that makes web requests (browsers, bots, scripts, malware etc) is free to send it's user agent string in whatever form it likes.
And while web browsers and bots send their user agents in a wide variety of ways, most are still fairly predictable... But when crawlers, scrapers, badly written scripts, and attackers come into the picture, things get pretty wild, very quickly!
Being able to spot these oddities can help you identify risky behavior on your site and help reduce fraud and security problems.
There are dozens of reasons a user agent can seem "weird" and it's a good indication if the traffic is legitimate or not.
User agents can have malicious fragments that attempt to inject SQL or execute commands.
Some fake user agents have restricted (or naughty!) bits in them. Don't show them to customers or staff.
Even the humble user agent string isn't immune to spammers. Don't show display user agents that have spam fragments in them.
It's likely that most user agent strings you've seen follow a pretty standard format; they open with a Mozilla/5.0 fragment, and then maybe some Windows NT or Android or Macintosh fragments, then a Chrome/, Firefox/ or Safari/ fragment and so on. Or perhaps there's an obvious fragment like Googlebot/2.1.
In other words, while fairly "standard" user agents are still a bit of a mess, they're not that crazy.. right?
Wrong
You'd be forgiven for thinking that, but actually, User Agent Strings can come in a literally unlimited number of formats, and since they are just strings of text, often generated by malfunctioning or incorrect code (at best) and malicious and dangerous code (at worst), if you are doing anything with user agents, it's important to handle them correctly.
You can also use user agent to glean extra information about who or what is sending the request; this can help you identify traffic which should be blocked or at least investigated further.
All of the paid API plans get full access to these fields to help you identify many, many types of problems with the user agents accessing your systems.
As well as checking them for problems, we also sanitize/normalize/fix user agents too.
Our user agent checks help you and your systems stay safe and free of junk user agents.
Use the API to identify malicious user agents, weird user agents, restricted user agents, and user agents with spam in them.
The API is free to use and easy to set up, so why not get started right now.
Do you have a question? Get in touch! We'd love to help you.