Our renowned RAM Analysis for Investigators class is now online, click here for details
As the Court case between the FBI and Apple has gone away (for now), I offer a cautionary tale. To back door or not to back door – that is the question.
The King and the Apple
Once upon a time there was a Kingdom in the west ruled by a powerful King. The people were affluent and happy and ate a lot of apples. In the eastern part of the Kingdom was a man who made locks. His locks were so good that most people in the land used them, his locks had made him a very wealthy man.
Although it was a generally good Kingdom, as with everywhere, there were those that would undermine the security of the Land, would break into houses and steal possessions of others. There were even those that hated the Kingdom and would stop at nothing to terrorise it and its people.
One day the King visited the lock maker. The King asked that the lock maker make him a special key that would let him enter the houses of those that would cause him trouble. He promised that the key would only be used to open the doors of the worst criminals. This, he said, would make the Land more secure. The lock maker loved the King and wanted to do all he could to help. He wanted everyone to be secure, so he made him the special key.
For a while the King used the key to open the houses of just the worst criminals but it worked so well that the King realised that it could be used to open the houses of anyone he didn’t like if he had more keys. The lock maker wasn’t happy but the King had said that it would make the Land more secure so he made lots of keys for all his enforcement militia.
Although most of the militia were loyal to the King there was one man serving who was also giving information to the criminals for money. He gave them a key. Now the criminals could open all the doors. The people heard about it and didn’t feel more secure, quite the opposite.
Meanwhile in a Kingdom far away in the east they had also heard about the lock maker’s amazing locks and had been buying them for many years. However, the Emperor heard about the special key that could open any of the locks. Knowing that it could be done, he tasked all his wisest subjects to find a way to make a special key. It didn’t take long. Now the Emperor could open all the locks, including the locks in the western land.
The King in the west heard that the Emperor could open all the locks, he didn’t feel so secure anymore.
Later, a young lock maker’s apprentice heard from the girlfriend of the friend of a person in the militia that you could make a key to open all the locks. The young man was very intelligent and in no time at all had worked out how to make the key. He believed that everyone should know how to do it, so he wrote to every newspaper in every town in the Kingdom so that anyone could make such a key.
Now, no one felt secure, not the people, not the criminals, not the terrorists, not the militia, not even the King. No one bought the wonderful locks anymore and the wealthy lock maker went out of business.
However, in the land in the East a new clever young lock maker made an incredible lock that no one could open. Everyone bought his locks and felt secure again.
A few months later the Emperor of the eastern land stood before the young lock maker. The Emperor explained that if he could make him a key to open the new locks, that it would make everyone more secure.
The King in the West learned that he could no longer open locks, but the Emperor could, and tragically choked on the apple he was eating and expired.
As I carry out a significant amount of OSInt work I often bump into the problem of needing to enumerate IP addresses. This can include knowing what my own external IP address is. Simply running ifconfig (or ipconfig in Windows) will provide my internal addresses but not the internet facing address from the router. This is especially important when trying to ensure that you are hidden from a target. It could be that I connect to a VPN or proxy elsewhere in the world but how can I be sure that my IP address is hidden?
A student on my recent Advanced OSI course related a story of a colleague researching a very dangerous group and suddenly realising that their VPN software had crashed and that their Police IP address was now visible in their targets logs – not good!
Their are loads of tools, especially Firefox plugins, that will report your IP and the IP of the site you are on, WorldIP is a favourite. However, I wanted to write a small program that would monitor my IP and report if it changes. I also wanted to be able to write a tool to do batch look ups of domains and IP’s and extract their Geolocation information.
I stumbled across freegeoip.net. It is a simple IP look up site but with an API. It allows 10,000 look ups per day for free which is more than enough (for most days!).
To use just type into your browser –
and it will return information about your own external IP address into a CSV file. Lovely! The results look like this…
You can also specify /xml, /json and /jsonp.
By adding a URL or IP address to the query will return the information about that address…
…and it returns…
184.108.40.206,US,United States,NY,New York,Somers,10589,America/New_York,41.33,-73.70,501
or if you specify /xml…
<Response><IP>220.127.116.11</IP><CountryCode>US</CountryCode><CountryName>United States</CountryName><RegionCode>NY</RegionCode><RegionName>New York</RegionName><City>Somers</City><ZipCode>10589</ZipCode><TimeZone>America/New_York</TimeZone><Latitude>41.325</Latitude><Longitude>-73.698</Longitude><MetroCode>501</MetroCode></Response>
To do this programmatically perhaps from a Shell script I can just use wget
Using this I can write a simple background tool that monitors my IP address and notifies me of any change. It will also be easy to have a tool which can be pointed at a text file of IPs or domains and returns all the information to me. That will save loads of time.
I’ll post the tools when I’ve done them.
I was teaching RAM analysis at the Swedish Police Academy this week, which included a segment on parsing out the MFT. This is an extraordinary capability that opens up a view of the disk to an investigator, which they may not have. Perhaps the RAM was taken but the plug pulled on an encrypted disk or maybe because of covert imaging considerations.
Each MFT entry is 1024 bytes which is taken up by the file name, accessed, modified and created dates and so on. However, if this data plus the file data is less than the 1024 bytes then the raw file data, the hex, is written to the MFT itself rather than out onto the disk somewhere.
The MFT parser is simple to run once you have an instance of Volatility running (see https://code.google.com/p/volatility/)
Python vol.py mftparser –f pathtoRAM >> pathtoaTEXTfile
When you view the rather verbose output from the Volatility MFT parser you stumble across entries that look like Fig 1:-
Here we can see a GIF image named AU_bg_TopMiddle.gif created way back in May 2005. The text output contains 3 columns: the virtual file offset addresses, the raw hex and the ASCII interpretation of the hex.
One of the students said it would be cool if we could ‘carve’ the original file out of the MFT result. Of course you could simply use Foremost, Photorec or a host of other data carvers on the RAM dump itself and the image would be found but it would have no file name, no metadata and be completely unattributable.
So, this seemed like a good idea to try.
I copied the $DATA chunk out of the text file, but being a text file it was completely unfriendly, dragging the other 2 columns with it. See Fig 2.
Next I manually deleted all the addresses, then all the interpreted ASCII to leave myself with the raw data. (Fig 3)
Then I fired up the awesome WINHEX and attempted to import it as ASCII-HEX but it was rather unhappy with the carriage returns and spaces. After about 20 minutes of faffing about I eventually managed to have the raw data sat in the Winhex window and it saved as a tiny, pointless GIF. But it worked!
That evening at the hotel I decided that a Python script was in order and a few hours later I had finished MFT2File.py. You can download it here.
Life is now much easier. Copy the chunk of data out of the MFT from below the $DATA line to the end of the Interpreted ASCII into a text editor like Notepad++ and save it in the same folder as MFT2File.py (to make life easier). Also, make a note of the original file name.
Next, open a command shell, ‘cd’ into the folder with the .py and text files in and run:-
First it will ask for the name of the original file. Make sure you at least get the extension right.
Second, it will ask you for the filename of the text file you made (and path if you didn’t put it in the same folder like I suggested). Fig 3
That’s it. The file will be magically recreated in the same folder as the .py file.
These files are small and some are text anyway like cookies and HTML file etc. However, it is great to see them as the original file and a fun project.
P.S.- Michael Ligh from the Volatility dev team just let me know that Volatility 2.4 will have a dump option to achieve this which is superb!
Whilst teaching my recent OSI course we had spent a good deal of time mapping the online infrastructure of a company using Maltego. The footprinting ‘machines’ are really superb and if you haven’t played with the tool go get it now!
Later in the day we were extracting company employee data from resources such as Data.com and LinkedIn and one of the students tried mapping the data with the Import option in Maltego. He had mapped employee name to office location and the map provided an immediate view of the approximate physical infrastructure with larger numbers of employees naturally oriented to HQ’s and small numbers to sub-offices. It was interesting to see. We did some standard research and the ‘map’ had been correct in ID’ing the primary HQ and sub offices.
Of course, the output is only as good as the data but this is where a tool called Jigsaw comes in (http://www.pentestgeek.com/tools/). Jigsaw was a business style social network where, if you uploaded your Contacts database you had access to the huge online repository. It became so good, especially in the US that it was bought by SalesForce and re-branded Data.com. The Jigsaw tool was incredibly good as you could extract vast amounts of information from the Jigsaw database on company employees so the data was obfuscated by SalesForce to make it fairly useless to the researcher. However, it still provides a partial name, job role, office location and other useful data if we are purely looking for sets of information.
The Jigsaw tool is no longer available to the public by can be found on Kali (www.kali.org). I won’t talk you through running it, its pretty self-explanatory but you start with simply running a search on the company of choice.
jigsaw -s BankofAmerica
Searching for Bank of America provided over 9000 employee records which I duly downloaded to a csv file. Next, do a ‘data’ import into Excel, comma delimited, and save as an xls file.
Next use the import tool in Maltego and map the Employee field to a Person entity, Department to the Shop entity and the City to the Location entity. When I tried to import the entire 9000 records, Maltego tried to generate over 28000 nodes and edges and simply fell over, however I re-imported selecting every 3rd record which worked fine.
During the import process you are asked to map columns to eachother. Map Person to Location and Person to Shop.
Once imported select the bubble view and interactive organic mode. This will have the effect of clustering related data together. What is interesting is that employees are naturally ‘drawn’ to their City and departments primarily located in those Cities are also attracted.
We can straight away see the largest Yellow node (Location) bang in the middle of the cluster map is Charlotte, essentially most employees in the database say they work there.
A quick check online shows that Charlotte is indeed the BoA HQ. The next ones are New York and the surprising locations of San Francisco, Miami, Plano and Wilmington. This helps us to ID at a glance the primary locations.
Next, the grey nodes are Departments. Again the map shows that most work, unsurprisingly in Finance and Administration followed by IT & IS, Support Marketing and Operations. This can really help us to visually map out the organisation, giving us an idea of the comparative sizes of departments.
I am going to do a little more work on clustering Department to Location to help us know where primary departments are located. Im not suggesting that this leaks anything particularly bad or dangerous but is an interesting view for a social engineering attack to begin. It could be that a company hides (or at least doesn’t actively publish) its Research department locations but this approach could identify it.
OK, I’ve spent another hour playing around and there is some interesting data you can get from this view. I mapped purely Location to Department and it was immediately apparent that I could quickly see where departments were and more importantly were not.
We can see that IT & IS are in virtually every office, however:-
…we can see that Human Resources departments only appear to be represented in about 8 primary locations. This would be vital information for a social engineer who could make a simple error by saying in a Phishing phone call that they were in HR in Atlanta, but its possible that there is no (or at least not large) HR department in Atlanta.
Interesting stuff, have a play and let me know your findings.
Having spent the best part of the last decade working on Live Forensic techniques I’ve begun to turn my attention to OSX. I’m an unashamed MacHead but have not spent much time thinking about ways to extract data from a live machine.
Tucked away in a SQL Lite table is a large list of ‘Recent Contacts’
Knowing who a suspect speaks to or emails can be very useful in an investigation and so I’ve started looking at the email system in OSX. The inbuilt email app, Mail is very widely used and connects to the OSX Address Book for the management of contact data. However, tucked away in a SQL Lite table is a large list of ‘Recent Contacts’, which contains the name and email address of recently contacted people who may or may not be in your standard contacts.
You can see this list by opening OSX Mail and browsing to Window – Previous Recipients. This opens a box with all the recent contacts, but apart from being able to add the contact to your main contacts, there is no way to export them.
I’ve written a small shell script to extract the name and email from the SQL table and pop them in a csv file for you.
The code is very simple, just 2 lines:-
echo 'First Name,Surname,Email Address' > ~/Desktop/recentcontacts.csv
This simply writes the column heads to a CSV file on your Desktop
sqlite3 -csv ~/Library/Application\ Support/AddressBook/MailRecents-v4.abcdmr 'select ZFIRSTNAME, ZLASTNAME, ZEMAIL from ZABCDMAILRECENT;' >> ~/Desktop/recentcontacts.csv
This opens the MailRecents SQL file and pulls out the first name, last name and email address, writing them to the CSV file on your Desktop.
For ease just drop the file somewhere, ‘cd’ to it and run – ./recentexport.sh
If it doesn’t run you might have a permissions issue so just type – chmod +x recentexport.sh
Hope its useful to you.
This tool stems from the need to extract unencrypted Skype chat from a RAM dump.
Its a bit old now and needs some work but people still have good results from it:-
1. Run Strings against your RAM dump
2. Run the Skypeex tool against the resulting Strings file
3. It will carve out all the Skype chat lines it can see as well as trying to find and extract all the Skype sessions and ‘orphan’ chats that have been created.
It’s interesting to note that the latter process even seems to find the ‘spam’ message sessions that you sometimes receive.
This has been tested on dump files from Windows XP2, XP3 and 7 with Skype 3.8 through 4.2.
Please do not hesitate to get in touch with ideas and improvements.
skypeex26 is designed for use under Python 2.6
For best testing results, have several Skype IM chats with friends and then image your RAM. On a windows box, use any tool to grab RAM (tested on Win XP SP2/3):
I recommend dumpit from Matthieu Suiche – http://windd.msuiche.net/
Run strings against the RAM image (e.g. Windows version can be found in Helix distro)
example: strings c:\ramdump.dd > c:\stringsout.txt
On linux box do:
strings ramdump.dd > stringsout.txt
Script usage –
from command shell – python skypeex.py – then, when prompted, simply provide the path to the strings output file.
The output files will be written to the folder where the script is run from. The output is a CSV file with chats (incl headers) and a txt file with extracted skype sessions and carved orphan chats. Please expect many duplicates and some false positives.
In the CSV file the ‘Timestamp’ column is the date and time of the message in UNIX time. Sorting on this column gives you a timeline of messages. I’m writing a UNIX time decoder but it doesn’t work yet.
The primary message content is in the ‘body_xml’ column.
ivMeta is a tool designed to extract useful forensic metadata from iPhone video. It was written by Robin Wood from digininja.org (Top pen tester) in response to some research I posted about finding useful tags and data inside the iphone .mov filetype. (Read that here)
Unzip the zip container and read the readme file. You know, the thing you never read because you can figure it out!
The tool will extract phone make, software version, created data, GPS data etc.
Drop us a line with any bugs.
First question, if you start a sentence with the word iPhone should you captialise the ‘I’, answers on a postcard please.
Second question came from a law firm that I often assist with digital forensics cases. When an iPhone is used to take a video and then distributed does it contain any device ID information that can be used to trace it back to the original phone?
The answer, somewhat surprisingly knowing Apple, appears to be no, I cannot find any reference to the serial number, IMEI or ICCID numbers within the file although it is possible that the data is there but obfuscated in some way.
Whether there or not, looking at iPhone movie data is very interesting. We are all used to the vast amount of metadata embedded within a photo but movies are a bit more of a dark area with not much written about it. The movies are based around the QuickTime file type that is well documented by Apple which can be found here – http://developer.apple.com/library/mac/documentation/quicktime/qtff/qtff.pdf
The filetype is awash with metadata, some which are used by default in the iPhone and many that are not. Although there does not appear to be anything to specifically identify the iPhone which shot the video there are some useful bits of data which could help. I have focused on a video shot by an iPhone 5 and then emailed out of the device.
The QuickTime structure is based around Atoms and Keys. Atoms are small 4 character tags such as ‘prfl’ for profile, ‘tkhd’ for the track header and many, many more. There are also keys that are of specific interest to us as they contain the primary metadata that we may want. The keys are in the ‘mdta’ atom and take the form of ‘com.apple.quicktime.author’, for example.
At offset 0x04 you come across the ‘ftyp’ atom which identifies the type of video to follow. The iPhone uses QuickTime and so the tag which follows is ‘qt’.
Next is the ‘mdat’ atom which I guess stands for movie data and contains the data related to the movie itself.
Next is the ‘moov’ atom which partly indicates that the movie came from a Mac platform, ie the iPhone. The ‘moov’ atom has a number of sub-atoms which brings us to the area we are interested in.
Once we pass all the obvious movie data we pick up a ‘keys’ atom which is then followed by metadata identified by the atom ‘mtda’. The entire section can be seen in the image below.
There are several interesting tags here.
©mak«Apple – This identifies that the movie came from an Apple manufactured device. Although this might sound obvious we might have a series of videos from a suspects computer that we think he may have taken. However, if he is an Android and PC user then this would reduce the likelihood that he created them.
©swr«6.1.4 – This is rather useful as it tells us the IOS software version that was installed at the time that the video was taken. Again, a scenario could be that a suspect accuses his co-defendant of shooting a video but we note that the co-defendants iPhone is running an earlier IOS version making it extremely unlikely that it was him.
©day«2013-05-27T21:38:21+0100 – This provides us with the time and date that the video was shot. Helpfully this date does NOT change when the file is moved, emailed or uploaded. This provides a solid line in the sand as to when the video was made. The time is also adjusted from UTC so we see the real world time it was created.
©xyz«+52.5461-002.6371+115.546 – This tag ‘@xyz’ provides GPS location data provided by the GPS chip in the phone. Although not delimited we can divide it up to provide:-
x – +52.5461
y – -002.6371
z – +115.546 – This appears to be the direction taken from the onboard compass.
This data depends on location data being turned on for Photos in the Privacy tab in Settings.
©mod«iPhone 5 – This is great, it doesn’t just tag the device as an iPhone but as an iPhone 5. Again this may help us to identify the phone in a case that shot a video. So we know the video was taken by an Apple iPhone 5 with firmware 6.1.4 on the 27/5/13 at 21:38:21 at a specific location. That’s not bad information.
All the information is then repeated using different tags as follows:-
So can we identify a specific device that shot a video? Not definitively no, however we may have a case where a number of phones are seized, perhaps a couple of Androids, an iPhone 3 and an iPhone 5. They may all have the same video on their phones showing illegal activity and be accusing one another of shooting it. In this case we may have sufficient metadata to pinpoint the culprit.
When I first started looking at this I assumed that it was a purely academic exercise as our normal forensic tools probably report this data but it seems not. A quick look in FTK with my test video only showed the Operating System dating, created, modified etc and not the embedded video created date. There was also no extraction of ANY of the metadata we have discussed, no model, firmware, GPS data, anything! Obviously you can manually work through the Hex to find the tags but it could easily be missed if we don’t know it’s there.
Hope that’s helpful to you?
After many years in the planning and execution (otherwise to be interpreted as a lackadaisical attitude to getting it done) we have at last launched a new web site. Its all based around WordPress to make it easier to edit and publish and I have been very impressed at the ease to which you can build and work on the site. The downside is that there is a very good chance that a new WordPress vulnerability will be found in the next year which will ensure that I end up as part of a vast Pharmaceutical BotNet. Every silver-lining has a cloud!
The site is rather simple and highlights the things we do to earn a living. It also, rather vitally, lists all the upcoming courses we teach along with a downloadable syllabus, costs and dates. Each course page also allows you to request booking of places. In the next few months I hope to be able to launch a calendar page with online payments too.
My blog will also move here from nickfurneaux.blogspot.com so bookmark the page for updates. Each blog post will also have a Tweet notification so please remember to follow @nickfx.
Thanks to Zane Clements for helping me put the site together.
I hope that the following 2 minutes that you spend perusing the site will be a good use of your life!