Fun with Python: Log Analysis and Indicators of Compromise (IoCs)
Introduction
In this installment of FwP, I again extend the log_anayzer.py script from previous articles. The first version of the script simply read a text file to find IPv4 addresses. It was then extended to write those addresses to .csv in the second version. I have extended it again to search for IPv4 Indicators of Compromise (IoCs).
I added another simple function, and a new entry to run it from the main menu. The function will open a txt file (i.e. an IoC list), read its contents, store all IPv4 addresses, and then compare the list against the list of IPv4 addresses found in another text file (i.e. a log file).
I also moved the call to clear the screen to its own function so that the results of the above function could still be visible when it returns to the main menu.
As always, this script is free to use under the GNU GPL. I make no guarantees or warranties, and the script is intended for educational purposes only. I am not responsible for any abuse or misuse of the script. Use at your own risk.
Preparation
Go back to the previous article and follow the preparation steps there.
Open the file log.txt created in the article for v2 and erase its contents. Then copy/paste the following into the file and save it:
apple dog river blue chair 192.168.0.24 star sky orange quiet dog mouse 10.0.45.67 grass key bottle guitar notebook lamp fire dog spoon clock 192.168.0.86 mountain flower tree shadow water dog mirror balloon paper 10.0.122.89 hat snow zebra ocean desert dog light thunder 192.168.0.145 cave rainbow
dog breeze forest pencil moonstone journey whisper dog stone coffee melon dog cactus ice 10.0.250.36 wind breeze drift berry 10.4.5.6 meadow rocket path 192.168.0.78 island cloud flame seed ladder canvas dog candle frost anchor twig 10.0.89.14 marble echo vine cherry storm dog river bird fog canvas river bench
dog arrow prism veil 192.168.0.199 jade dream arrow harp nest dog glove cedar honey quilt log 10.0.34.78 stream sail crystal timber scarf glow cave flame spruce clay coral rust dog fog sunrise trunk 192.168.0.123 velvet 192.168.1.2 ember grove amber dog lamp pearl frost bridge lily shadow ripple coin drift
stone dog 192.168.0.36 breeze leaf chair frost 10.0.78.22 wing river anchor bench dog blossom 192.168.1.3 path crystal moss glove 192.168.0.144 ladder dusk tree plank dog flame glow veil snow harp cedar twig log stream charm root echo bird dog bark 10.0.13.88 dream
mountain frost dog paper canvas shadow ladder glass lamp sunrise 192.168.0.92 ocean honey coral cedar dusk dog ivy quilt timber rust wave stone flame sail branch breeze fog berry star log moon dog coral spruce arrow ripple fog flame drift 10.0.56.34 nest frost
dog wheel echo shade vine amber stone frost tree 10.0.211.9 whisper hat charm ladder river bark flame pearl dog paper coin drift meadow mirror rust shadow dog 192.168.0.199 ember dusk root star glimmer twig crystal dog bench tide light 10.1.2.3 wing veil ember plank moon dog
Now add a new text file to the folder with the other files and name it “iocs.txt”. Add the following IP addresses to the new text file:
192.168.1.2
192.168.1.3
10.1.2.3
10.4.5.6
10.10.10.10
Open Notepad and copy/paste the script below. Name it “analyzer_v3.py”, and make sure to save it as a .py script by selecting All Files in the dropdown for Save type as in the Save menu.
The Code
The Code
import os
import re
app_running = True
log_file = open("log.txt")
log_strings = log_file.readlines()
def find_ipv4(n):
ipv4_list = re.findall(r"(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})", str(log_strings))
write_to_csv(ipv4_list, n)
def find_iocs():
ioc_file = open("iocs.txt")
ioc_strings = ioc_file.readlines()
iocs_found = []
iocs_not_found = []
ipv4_list = re.findall(r"(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})", str(log_strings))
ioc_list = re.findall(r"(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})", str(ioc_strings))
for x in ioc_list:
if x in ipv4_list:
iocs_found.append(x)
else:
iocs_not_found.append(x)
print("\nIOCs found: ",iocs_found)
print("IoCs not found: ",iocs_not_found,"\n")
menu()
def write_to_csv(ips, n):
unsecured_list = ""
for x in ips:
unsecured_list += (x + ",")
match n:
case 0:
ip_file_u = open("unsecured_list.csv", "a")
ip_file_u.write(unsecured_list)
print("Your file of unsecured IPs is being generated. Use with caution.")
app_running = False
case 1:
ip_file_s = open("secured_list.csv", "a")
secured_list = unsecured_list.replace(".","[.]")
ip_file_s.write(secured_list)
print("Your file of secured IPs is being generated. Use with caution.")
case _:
print("Error: Unknown. How did you get here?")
quit()
def clear_screen():
os.system('cls' if os.name == 'nt' else 'clear')
def menu():
ui = ""
print("******** analyzer ********")
print("1) Secured List ")
print("2) Unsecured List ")
print("3) Search for IoCs ")
print("q) Quit ")
print("**************************")
while app_running:
ui = input("_: ")
match ui:
case "q":
app_running == False
quit()
case "1":
find_ipv4(1)
case "2":
find_ipv4(0)
case "3":
find_iocs()
case _:
print("Error: command not recognized.")
clear_screen()
menu()The Script in Action
Open a Command Prompt and navigate to the folder.
Type the command:
python analyzerThe main menu should appear. Notice the new menu option: 3) Search for IoCs
Input 3 into the prompt and press Enter:
The function will run and output two lines showing list of the IoCs that were found in the log file, and the IoCs that were not found. It will then display the menu again.
The Code Analyzed
The new function is simple.
First, it opens the .txt file containing a list of IoCs:
ioc_file = open("iocs.txt")
ioc_strings = ioc_file.readlines()It creates two list variables for storing the IPv4 addresses found in the two .txt files that will be compared:
iocs_found = []
iocs_not_found = []It then scans both files for IPv4 addresses using regex and stores the results in the variables:
ipv4_list = re.findall(r"(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})", str(log_strings))
ioc_list = re.findall(r"(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})", str(ioc_strings))
Next, it loops through the IoC list, compares it to the log file’s list, and appends any found or not found IPs to the associated list variables:
for x in ioc_list:if x in ipv4_list:iocs_found.append(x)else: iocs_not_found.append(x)Finally, it prints the results to screen and returns to the main menu:
print("\nIOCs found: ",iocs_found)
print("IoCs not found: ",iocs_not_found,"\n")
menu()Summary
While writing a previous article on APTs, I got the idea to extend this script to search for IoCs. Threat hunting is a common and important task for any defender, and expensive tools are not always available. So, I wanted to see how Python might be leveraged. Next time, I’ll take a look at combining this script with another script that searches for files in multiple directories to see how they can be utilized together to search for other types of IoCs.
Again, as with all my scripts on the site, this script is free to use under the GNU GPL. I make no guarantees or warranties, and the script is intended for educational purposes only. I am not responsible for any abuse or misuse of the script. Use at your own risk.
Thank you for reading and have fun.
Daily Cuppa
Today’s cup of tea is Organic Earl Grey provided by Equal Exchange.
Fair Trade, organic, and bursting with bergamot!
If you enjoyed this article fee free to buy the author a cup of tea.