Yep, another Pwned Passwords post! This one brings the total to 3, and it now makes up the entirety of my posts here.
A couple of days ago, Troy Hunt released support for NTLM hashes for his Pwned Passwords dataset. This is really cool because it allows us to check live Active Directory hashes from ntds.dit (located under C:\Windows\NTDS on Domain Controllers). However, there’s a little bit of work involved in getting to that step, so I thought I would put together this quick little guide.
Extracting Hashes from NTDS.DIT
So, how do we get the hashes out of that ntds.dit database?
There’s actually a few different ways, but the easiest I found from some cursory searching was to use Michael Grafnetter’s amazing PowerShell tool, DSInternals. If you don’t want to go about doing a manual install of it, you can get it straight from the PowerShell Gallery, with quick instructions here.
Once you’ve got it installed, the next step is to make a copy of ntds.dit. If you try directly from C:\Windows\NTDS, you’ll notice the file is locked. Again, there’s a few ways around this, but a quick way is to do a shadow copy of the system drive, copy it from the shadow copy, and then delete the shadow copy, like so –
Then, you need the SYSTEM registry hive. Luckily, this can be directly saved, by running –
reg save HKLM\SYSTEM <YOUR OUTPUT PATH HERE>
Now that we’ve got both of those files, we can start extracting hashes!
With the system extract and ntds.dit copy in hand, fire up PowerShell, it’s time for DSInternals!
Running the following will extract the hashes in Hashcat format (necessary for the next step) –
$key = Get-BootKey -SystemHivePath '<PATH TO SYSTEM HIVE HERE>' Get-ADDBAccount -All -DBPath '<PATH TO NTDIS.DIT HERE>' -BootKey $key | Format-Custom -View HashcatNT | Out-File <OUTPUT PATH HERE> -Encoding ASCII
Note: If you receive any errors about your ntds.dit file being invalid, this is a good guide to fix it. Essentially, you just need to run this command on the ntds.dit copy from a Domain Controller –
esentutl /p <PATH TO NTDS.DIT> /8 /o
Comparing the Hashes
Now that we have a neat little hash extraction, we can get to comparing our users’ passwords against the Pwned Passwords dataset. If you don’t have a copy of the NTLM hashes, you’ll need to grab them from here.
In Troy’s original post about adding NTLM hashes, he mentioned a tool to compare the passwords from Pwned Passwords with extracted passwords. This tool is available here. Unfortunately, as it builds a hashmap from a large text file, it’s not that fast.
Once downloaded, all you need to do is specify the function, and supply it with the path to the output file from the previous step, alongside the path to the pwned passwords dataset. The syntax and an example of the output can be seen below –
Note: The time is not representative of the full dataset, this was an example where the hash dictionary contained only the single hash. When using the actual dataset, it will take more than a few minutes (on average) per user.
So, now we know which users are currently using a breached password!
A Look at Faster Comparisons
While the tool above is certainly good, it just doesn’t scale too well. There’s also another great looking tool here, but again, the issue is speed. Definitely cool though! I decided to have a look at changing my original binary search for the original Pwned Passwords dataset (read my post here if you want more information on that).
As binary search requires a sorted input file, and the dataset is ordered by frequency, I went about sorting it myself. The sorting took about 40 minutes, but that’s on some pretty old hardware, and a pretty basic set up. It would be great to have the hashes available sorted by hash, but it’s not too bad if you have to do it yourself. I’d publish the sorted dataset but I don’t think my Australian internet would ever finish the upload.
In any case, I’m sure you could find a better sorting algorithm out there, but I used this and grabbed the necessary Priority Queue and Heap classes from the author’s GitHub. You’ll also need to modify it to ensure the frequency isn’t included when sorting.
You can find my code at my GitHub. Here’s an example of the syntax and the output –
There’s also a release available if you don’t want to compile it yourself.
I think the addition of NTLM hashes to the Pwned Passwords family will offer up even more use cases for the data. I’ve said it before, but it really needs to be said that Troy’s work is invaluable. He provides a great service to the infosec community.
I believe that the combination of this, alongside something to check against password changes (like my Pwned Passwords API checker, or the several others out there), will greatly help security folk, and hopefully help users see just how insecure some passwords are.
If you’ve made it this far, congratulations. I hope you have a great rest of your day/night! 🙂