A project I’m involved with that’s still in it’s early stages is the development of an automated, intelligent security environment that consists of:
- Firewall, router and switch configuration management (existing in-house developed software).
- Asset management and vulnerability scanning (Metasploit and OpenVAS).
- Malware detection and analysis (analysis performed by Cuckoo).
- Intrusion detection and analysis.
The last two are the final pieces to our puzzle, and given the data has high:
- Variety: applicance, application and server logs.
- Velocity and Volume: dozens of networks across the globe, hundrens of servers, 1000+ databases, 1000+ applications.
… it’s really a no brainer to use Hadoop as the storage framework. To dip our toes into the newfound waters of big data, whilst also evaluating a solution that could prove useful in piecing together our environment, I pieced together a network analysis server consisting of Snort, Hadoop, Pig and PacketPig. Continue reading “oinkception”
