Too many times we have situations we suffer from random network connection / latency problems which might be caused by a huge amount of traffic sent over the network or even a problem directly on the specific endpoints.
If you have such kind of problems, in the most of the cases it’s mandatory to capture network traffic on the affected endpoints or even better also on a network device in the middle.
The most popular tool to analyze network captures (.pcap files) is Wireshark.
There are many good tutorials in the Internet which shows good ways to understand and finding the root cause for such Incidents.
I am using Wireshark since more or less 4 years now and after that time and also after many network trace files I’ve looked in there are some features which are not included or hard to get out of Wireshark (e.g. Top Talker Conversations over time, Number of devices between the conversation endpoints).
So I started to think about a way to collect the network packet data by using Splunk. After several weeks checking which will be the best way I’ve decided to use the CSV functionality.
Wireshark includes several sub components including the tshark component which is able to convert the .pcap file to a Splunk readable CSV file.
I came up with the “PCAP Analyzer for Splunk” which can be downloaded from Splunk App page:
https://splunkbase.splunk.com/app/2748/
How to get started?
“PCAP Analyzer for Splunk” requires a full splunk instance running on *nix / Windows systems and Wireshark needs to be installed.
In summary the most important requirements are the following:
- Splunk instance with SPLUNK_HOME variable defined.
- Wireshark installed (on Windows: Wireshark needs to be located at %programfiles%)
- Confirm that your user has all administrative privileges for the folders “$SPLUNK_HOME/etc/apps/SplunkForPCAP/bin/” and for your Wireshark folder.
What’s next?
As soon as you have installed the “PCAP Analyzer for Splunk” on your Splunk instance the next step is to define a folder on your system which should be monitored for “*.pcap” files.
You can define this folder in the WebUI via Settings –> Data Inputs –> PCAP File Location.
In this configuration panel you have to define a name and a path.
Lets make an example and use as name “myfolder” and as path “/var/tmp/”.
With this configuration a new folder will be created automatically at “$SPLUNK_HOME/etc/apps/SplunkForPCAP/”. It is the “local” folder which includes an “inputs.conf” file.
How to get my first “.pcap” file into Splunk?
If you can confirm that the above steps has been done successful, the next step is to put a “*.pcap” file in your folder you have defined (e.g. /var/tmp/).
Now the rest is done by the app.
“PCAP Analyzer for Splunk” checks every 3 minutes if a new pcap file has been added to your folder.
If it finds a new pcap file it will be converted and indexed automatically into Splunk.
You can open a panel in the application and you should find your pcap file in the dropdown menu.
Good things to know!
- As soon as your “*.pcap” file is processed, it will be moved to “$SPLUNK_HOME/etc/apps/SplunkForPCAP/PCAPConverted”. No need to worry 🙂
- Your files will be converted to a CSV file which are created in the folder “$SPLUNK_HOME/etc/apps/SplunkForPCAP/PCAPcsv”. You can delete those file after the indexing is done.
Summary
The tool helps me to understand details about each pcap file and you can even use it with the free version of Splunk on your workstation. When I receive a file to analyze, it will take 2 minutes to have the file converted into Splunk. Follow those steps and everything is set up properly.
If you want to know more about it, join my Slack Workspace or send me an email.
Stay up-to-date and subscribe to my Newsletter!
I want to analyze static pcap file, How can i do that?
Hi, the app is quite cool.
if i may suggest setting the selection of the index to search into a macro for easy adaptation? Also the placementof the location for the converted pcap and the csv is a bit unfortunate if you tend to do a fair amount of pcap ingestion (i tried 500mb).
Also the selection of fields to be indexed would be cool if it were in the input definition. But also maybe it would be easier to just directly get the csv from stdout.
Thank you for your feedback! I am working on a new version at the moment.
Good Morning,
I am not an avid user of anything Apple. However, I find myself having to use a MacBook to analyze .pcap. I ran across your program which is amazing by the way, but am having a few issues.
As a windows user, I know that add ons need to be in the specific folder that the other program is running from. Example: World of Warcraft add-ons need to be in the WoW folder.
Does PCAPAnalyzerforSplunk need to be in the SPLUNK folder in order to work properly?
Daniel, thanks for creating this tool. It looks like it might be useful to me but I could not get it to work. I this error mentioned in Messages. “Unable to initialize modular input “pcap” defined in the app “SplunkForPCAP”: Introspecting scheme=pcap: script running failed (exited with code 1)..”
When I go to settings Data Inputs, I don’t see a data importer for PCAP.
And when I go to the App Dashboard examples, I see examples from the simple_xml_examples I installed.
Can you help me?
Daniel,
Great work! You’ve spent a lot of time on this and the dashboards look great. I’m using this to teach both Wireshark and Splunk to some students, but I’m also going to use this in production.
Well done, sir.
Thanks,
Hank