URLhaus API
URLhaus offers an API to both, receive (download) and submit malware URLs from the URLhaus database. The API is documented below.
Database dumps
We provide various URLhaus database dumps. Depending on your need, you can choose between a full dump (containing URLs that are either actively distributing malware or that have been added to URLhaus within the past 90 days), a dump of active malware distribution sites only or a dump of any URL added to URLhaus within the past 30 days. You can choose between CSV and JSON format. The dumps are generated every 5 minutes. Please do not fetch it more often than every 5 minutes.
Full database dump, contains URLs that are either actively distributing malware or that have been added to URLhaus within the past 90 days:
Download JSON (zip compressed)
Recent database dump, contains recent additions (URLs) only (past 30 days):
Download CSV (recent URLs only)
Download JSON (recent URLs only)
Active database dump, contains online (active) malware URLs only:
Daily MISP Events
You can download URLhaus IOCs as daily MISP events. New MISP events get generated at midnight. Plese do not try to fetch them before 00:15 UTC.
DNS Response Policy Zone (RPZ)
By using an DNS Reponse Policy Zone (RPZ), also known as DNS firewall, you can block the resolution of certain domain names on your DNS resolver. URLhaus extracts the domain names from malware URLs and offers them in an RPZ dataset. More information about DNS RPZ can be found on dnsrpz.info.
The RPZ zone file gets generated every 5 minutes. Please do not fetch it more often than every 5 minutes.
Note
To reduce the amount of false positives, URLhaus RPZ does only include domain names associated with malware URLs that are either active (malware sites that currently serve a payload) or that have been added to URLhaus in the past 48 hours. In addition to that, Tranco Top 1M are excluded from the RPZ dataset.
Snort / Suricata Ruleset
If you are using a network intrusion detection and preventation systems (IDS / IPS) like Snort or Suricata (or any other IDS that supports the Snort / Suricata Ruleset format), you may use the URLhaus IDS Ruleset to identify network traffic towards known malware URLs. The ruleset will only trigger on the extact URL in a HTTP stream (HTTP GET request).
Note
Due to the vast amount of malware URLs tracked by URLhaus, the Snort / Suricata ruleset does only include malware URLs that are either active (malware sites that currently serve a payload) or that have been added to URLhaus in the past 10 days. If you would like to watch out for offline malware URLs too, you should use a different tool than Snort or Suricata.
The IDS ruleset gets generated every 5 minutes. Please do not fetch it more often than every 5 minutes.
If you are using Suricata IDS, there is a dedicated ruleset available that is using sticky buffers, which comes with a performance increase compared to the standard ruleset above:
host file (domains only)
Some commercial and open source security software (such as Pi-hole) can block access to domain names based on the host file format. For this purpose, URLhaus offers a list of domain names associated with malware URLs below.
Note
To reduce the amount of false positives, URLhaus host file does only include domain names associated with malware URLs that are either active (malware sites that currently serve a payload) or that have been added to URLhaus in the past 48 hours. In addition to that, Tranco Top 1M are excluded from this dataset.
Plain-Text URL List (URLs only)
The Plain-Text URL List is a dump of all malware URLs known to URLhaus. It does not contain anything else than one URL per line, which is useful if you want to use the URLhaus dataset as an IOC (Indicator Of Compromise). You can match them against certain log files of your security permieter, for example web proxy logs. You may also use it as a blocklist with a low false positive rate.
The Plain-Text URL List gets generated every 5 minutes. Please do not fetch it more often than every 5 minutes.
As the database dump (Plain-Text file) mentiond above is growing in size, some commercial security solutions are not able to fetch it. URLhaus therefore provides the following sub-sets:
URLhaus database dump (Plain-Text) containing recent additions (URLs) only (past 30 days):
Download Plain-Text (recent URLs only)
URLhaus database dump (Plain-Text) containing only online (active) malware URLs:
Collected Payloads (CSV)
URLhaus regularely checks the content served by malicious URLs that are known to URLhaus. This CSV contains all payloads collected by URLhaus, identified by a hash (MD5 / SHA256 hash). Please consider that not all payloads are malicious. As a matter of fact, a URL can e.g. serve any content once it has been cleaned up.
URLhaus ClamAV signatures
URLhaus generates a ClamAV signature database which gets updated once per minute. This allows you to add almost real time detection of malware distribution sites (e.g. such ones being used by Emotet/Heodo) on your email gateway / spam filter. As the signature file only contains active malware distribution sites or such that have been added to URLhaus in
Download ClamAV signature database
ClamAV SHA256 checksum file:
If you plan to implement the URLhaus ClamAV signatures, please ensure that you update the signature file once per minute to receive the best protection / spam catch rate. You can do so by setup a cron that executes the following script every minute:
I do also recommend you to either REJECT or QUARANTINE any email message that contains a link blacklisted in the URLhaus ClamAV signature file. In case you decide to REJECT emails containing blacklisted urls, you are free to put the following link into the error message you send back to the user:
Further information about the implementation of ClamAV can be found on the following websites:
Submit malware URLs
Collecting and maintaing a list of malware URLs means a lot of work. I therefore appreciate any submissions from 3rd parties like security researchers, SOC analysts or vendors to URLhaus. If you would like to submit malware URLs to URLhaus, there are two ways to do so:
- Manual submissions through the URLhaus web interface (note: you need to authenticate yourself with your abuse.ch account)
- Automated / bulk submissions through the URLhaus API (as documented below)
Note
For security reasons, and as I want to keep the URLhaus database clean from false positives, anonymous submissions are not accepted. If you want to submit a malware URL to the URLhaus database, you must authenticate yourself with your abuse.ch account. But don't worry, if you don't want to be associated with a submission, you can hide your abuse.ch handle by setting the option / flag anonymous report. While URLhaus will know where the submission came from, the source of the submission will not be shared with anyone.
Submissions via web interface
You can use the web interface to submit a malware URL to URLhaus. In order to do so, you will need to login with your abuse.ch account. Please consider that your abuse.ch handle will be public visible unless stated otherwise (by selecting the option anonymous report).
Submissions via API
There is a web API you can use for automated or bulk submissions. You can call the API through Python or your prefered scripting language.
To submit a malware URL to URLhaus through bulk API, you must send a POST request to https://urlhaus.abuse.ch/api/. The post request must contain the following fields (JSON):
| anonymous | If set to 1, your submission will be anonymous (required) |
|---|---|
| submission | List of URLs (required) |
| URL | URL you want to submit (required) |
| Threat | Threat (required, must be malware_download) |
| Tags | Tag. Allowed characters: [A-Za-z0-9.- ] (optional) |
In addition, your POST request must contain the Auth-Key field, containing your personal Auth-Key. If you don't have an Auth-Key yet, you can get one at the abuse.ch Authentication Portal.
If you want to send malware URLs to URLhaus using python, you can find a sample script here:
More sample python scripts showing how to interact with the URLhaus bulk API are available here:
Submission Policy
URLhaus is currently only collecting websites (URLs) that are directly being used to distribute malware. Please note that any other submissions will be ignored / deleted from URLhaus.
Before you start to submit URLs to URLhaus, I encourage you to read the following submission policy:
- Active malware distribution sites: Please ensure that you only submit active (online) malware distribution sites that are currently serving a payload (please see the definition of payload below). Malware URLs that are down and / or have already been cleaned should not be submitted to URLhaus.
- Payload: A payload can be any file (executable, script, document) that harms or infect a computer once downloaded and executed. Some examples: Windows executables, Office documents, PowerShell scripts, Bash scripts, hta, ELF.
- URL shorteners: Any URL submitted to URLhaus must host an active malware payload. Redirection sites or URL shorteners (e.g. bit.ly) that are just used for redirection and that are not hosting any payload should not be submitted to URLhaus.
- Adware is not Malware: Unlike Malware, most common Adware (aka Potential Unwanted Programs - PUPs) do need some sort of user interaction. In many cases, they also come with a licences agreement that the user has to accept and that is more or less transparent with regads to what the Adware does. Please refrain from submitting URLs to URLhaus that are distributing Adware.
- Phishing and Phishing kits: Phishing sites or websites that are hosting a phishing kit should not be submitted to URLhaus (Phishing is not Malware). If you would like to report phishing websites, you may want to report them to AWPG, PhishTank or Netcraft.
- Automated submissions: Should you decide to make automated submissions to the URLhaus API, please ensure that your script has implemented proper URL verification. Please also ensure that you do not submit any private IP addresses (RFC1597) or any IP addresses that are used for any other special purpose (RFC6890).
- Exploit kits: Websites that are hosting an exploit kit should not be submitted to URLhaus unless the submitted URL serves the final payload.
- Geo IP filter: Some malware distribution sites may use a Geo IP filter to restrict the download of the payload to a specific country. You can tell URLhaus about this restriction by using the tag geofenced and a tag with the three letter NATO country code (e.g. GBR for Great Britain). URLhaus will then try to fetch the payload using an IP address from the specified Geo location.
- Duplicates: To avoid duplicates and ensure that the malware sites tracked by URLhaus can be properly used as IOC, please make sure that you submit URLs to URLhaus as you see them on the wire, returning
HTTP 200 OK. For example,http://evil.tld/91BOYI/oamo/USwould becomehttps://evil.tld/91BOYI/oamo/US/(note the tailing /) andhttps://evil.tld?thisisbad=1would becomehttps://evil.tld/?thisisbad=1(note / after.tld).
Note: Should you repeatedly violate the submission policy documented above, your account may get banned from URLhaus.
Your Account
API for automated bulk queries
If you would like to query URLhaus for e.g. an URL or malware sample in an automated way, there is a dedicated API available for this purpose. It also allows you to download a specific malware sample or daily batches: