Pwned Passwords and NTLM Hashes!

Introduction

Yep, another Pwned Passwords post! This one brings the total to 3, and it now makes up the entirety of my posts here.

A couple of days ago, Troy Hunt released support for NTLM hashes for his Pwned Passwords dataset. This is really cool because it allows us to check live Active Directory hashes from ntds.dit (located under C:\Windows\NTDS on Domain Controllers). However, there’s a little bit of work involved in getting to that step, so I thought I would put together this quick little guide.

Extracting Hashes from NTDS.DIT

So, how do we get the hashes out of that ntds.dit database?

There’s actually a few different ways, but the easiest I found from some cursory searching was to use Michael Grafnetter’s amazing PowerShell tool, DSInternals. If you don’t want to go about doing a manual install of it, you can get it straight from the PowerShell Gallery, with quick instructions here.

Once you’ve got it installed, the next step is to make a copy of ntds.dit. If you try directly from C:\Windows\NTDS, you’ll notice the file is locked. Again, there’s a few ways around this, but a quick way is to do a shadow copy of the system drive, copy it from the shadow copy, and then delete the shadow copy, like so –

Then, you need the SYSTEM registry hive. Luckily, this can be directly saved, by running –

reg save HKLM\SYSTEM <YOUR OUTPUT PATH HERE>

Now that we’ve got both of those files, we can start extracting hashes!

With the system extract and ntds.dit copy in hand, fire up PowerShell, it’s time for DSInternals!

Running the following will extract the hashes in Hashcat format (necessary for the next step) –

$key = Get-BootKey -SystemHivePath '<PATH TO SYSTEM HIVE HERE>'
Get-ADDBAccount -All -DBPath '<PATH TO NTDIS.DIT HERE>' -BootKey $key | Format-Custom -View HashcatNT | Out-File <OUTPUT PATH HERE> -Encoding ASCII

Note: If you receive any errors about your ntds.dit file being invalid, this is a good guide to fix it. Essentially, you just need to run this command on the ntds.dit copy from a Domain Controller –

esentutl /p <PATH TO NTDS.DIT> /8 /o

Comparing the Hashes

Now that we have a neat little hash extraction, we can get to comparing our users’ passwords against the Pwned Passwords dataset. If you don’t have a copy of the NTLM hashes, you’ll need to grab them from here.

In Troy’s original post about adding NTLM hashes, he mentioned a tool to compare the passwords from Pwned Passwords with extracted passwords. This tool is available here. Unfortunately, as it builds a hashmap from a large text file, it’s not that fast.

Once downloaded, all you need to do is specify the function, and supply it with the path to the output file from the previous step, alongside the path to the pwned passwords dataset. The syntax and an example of the output can be seen below –

Note: The time is not representative of the full dataset, this was an example where the hash dictionary contained only the single hash. When using the actual dataset, it will take more than a few minutes (on average) per user.

So, now we know which users are currently using a breached password!

A Look at Faster Comparisons

While the tool above is certainly good, it just doesn’t scale too well. There’s also another great looking tool here, but again, the issue is speed. Definitely cool though! I decided to have a look at changing my original binary search for the original Pwned Passwords dataset (read my post here if you want more information on that).

As binary search requires a sorted input file, and the dataset is ordered by frequency, I went about sorting it myself. The sorting took about 40 minutes, but that’s on some pretty old hardware, and a pretty basic set up. It would be great to have the hashes available sorted by hash, but it’s not too bad if you have to do it yourself. I’d publish the sorted dataset but I don’t think my Australian internet would ever finish the upload.

In any case, I’m sure you could find a better sorting algorithm out there, but I used this and grabbed the necessary Priority Queue and Heap classes from the author’s GitHub. You’ll also need to modify it to ensure the frequency isn’t included when sorting.

You can find my code at my GitHub. Here’s an example of the syntax and the output –

There’s also a release available if you don’t want to compile it yourself.

Final Comments

I think the addition of NTLM hashes to the Pwned Passwords family will offer up even more use cases for the data. I’ve said it before, but it really needs to be said that Troy’s work is invaluable. He provides a great service to the infosec community.

I believe that the combination of this, alongside something to check against password changes (like my Pwned Passwords API checker, or the several others out there), will greatly help security folk, and hopefully help users see just how insecure some passwords are.

If you’ve made it this far, congratulations. I hope you have a great rest of your day/night! 🙂

Checking for Breached Passwords in Active Directory – Using k-Anonymity!

I’d like to preface this post by saying that I 100% understand concerns about using an external API, even when sending it just a small amount of unusable information. The possibility of compromise and subsequent infection on Domain Controllers is a true security risk and it is totally acceptable to not want to take that risk. For those not wishing to use an external API at all, I wrote an original post on checking breached passwords with AD, that works entirely offline with downloaded hashes of Troy Hunt’s Pwned Passwords – you can read about that project here.

Introduction

Last year Troy Hunt released a freely searchable database of previously breached passwords. The available data was pretty much a godsend for anyone looking to verify that their users aren’t using breached passwords, with the ability to query for passwords through plaintext or as SHA1 hashes via API, as well as the ability to download the entire data set to use offline. A little after the release of this information, I wrote a DLL for use in Active Directory environments. However, that tool required users to download the massive lists of passwords so that it could be queried over the network. While it requires no external service interaction, it still requires downloading and searching through gigabytes of files.

Continue reading “Checking for Breached Passwords in Active Directory – Using k-Anonymity!”

Checking for Breached Passwords in Active Directory

Edit: I have now overhauled the blog post and essentially recreated PwnedPasswordsDLL to run on-premises,
      and return results very quickly. Information regarding set-up and the new release can be
      found below. Changes have now been pushed to GitHub and are available for use.

Introduction –

In simplistic terms, PwnedPasswordsDLL will check a requested Active Direvtory password change against a local store of over 330 million password hashes. If the hash is found in the breached passwords, the requesting password is rejected. This entire process takes ~1 second against over 330 million previously breached password hashes.

Now on to a more technical explanation.

A little known setting through LSASS on Windows PCs is the ability to call a custom filter DLL when a password change request is received. The utilisation of this resource was commonplace back in the Windows 2000 days, but it’s now seldom used; many sysadmins are happy enough using totally third-party solutions or the in-built Group Policy settings.

The concept of a 100% customisable password filter intrigued me, and with Troy Hunt’s new freely searchable database of pwned passwords, I decided to look at setting up a filter DLL to call a local store of the breached passwords to check the prospective password change.

I discovered that writing such a DLL was a very simple process, most of the actual handling of the DLL is done internally, all that needs to be done by an author is to create a function that returns a Boolean expression based around whether or not conditions are met for the requesting password. The result of my research is PwnedPasswordsDLL.

In calling the DLL, the requested password is converted to an SHA1 hash through the Crypto++ library. This SHA1 hash is then passed to a binary search algorithm I wrote that searches the three HaveIBeenPwned breached passwords text files. If the hash is found, a false boolean is returned (i.e. telling LSA that the calling password failed to meet the required password policy).

If you’re willing to give it a whirl, you can find the code here.

If you want to compile for yourself, check out the Compiling the Code section below. If you want to download a Release version, look at the Implementing the DLL section below.

Here is an example of the DLL in use –

Continue reading “Checking for Breached Passwords in Active Directory”