Archive for : January, 2015

post image

What is Elastic Search | Log Stash | Kibana or “ELK”

What is Elasticsearch, Logstash, Kibana or “ELK” and how can I use it as a security professional?

kibanaELK is at it’s most basic a data processing, data storage and data visualization software trio.  Logstash provides automated ingestion of  of text data allowing for a limited amount of data normalization and standardization.  Elasticsearch is the data storage solution called a noSQL software which can process large amounts of data using a REST API. Kibana is the data normalization software that allows for a user to manipulate the data into “widgets” and set up different views to analyze the data.

When I first started to explore the ELK structure and it’s potential I was  unable to find a good set of explanations and use cases describing how I could use it.  That being said, I have learned a great deal about the use of ELK in the security space and I’m sharing it here:

Example use cases scenarios for Security:

  • Review Weblogs feeds to analyze data real time
  • Setup up metric to visualize IT Security operational data (including SOC operations)
  • Use timeline data (Volatility timeliner and Log2timeline) to view forensic data from multiple systems into one view

Logstash is versatile in that it can work on Windows, Linux or mac OS.  So, you can pick your favorite flavor of OS and install.  The technical level required to us ELK can be minimal.  Using the software out of box is advantageous as it has the capability with logstash to ingest those files and spit them into elasticsearch. Then Kibana seamlessly integrates into elasticsearch without any major configuration efforts.

If this has peaked your interest you can download the following files to get started here: http://www.elasticsearch.org/overview/elkdownloads/ 

Leave comments on how you use ELK as a security professional.

Time to start fresh, with a new blog site.

Wow… It has been awhile and the site is still a mess.

Over the last year I haven’t contributed to this blog the way I had originally anticipated and expected to.  My job has changed, I travel a lot more and have trouble blogging about IT operations as I have in the past.  Currently all the old blog posts are down… I’m sorry, but the transition to the new site didn’t convert my old blog posts into this site.  I can say that they are coming back as soon as I find the time to convert them into this site. I feel that the content may be old, but relevant content will get pulled down to this site.

As I mentioned, my job and interests have changed.  What does that mean?  That means I’m now focusing on IT Security – Incident Response / Management / Forensics.  I have roadmap of blog entries I’m currently looking to start build with the following outline.

  1. Information Security
  2. Incident Response
    1. Forensics
    2. ProActive Incident Response
    3. Tools
    4. Programming your own tools
  3. Big Data / Visualization
    1. Kibana / LogStash / Elastic Search
    2. Orange Canvas
  4. Operations
    1. WIndows
    2. Linux
    3. Mac OS
  5. Programming
    1. VBS
    2. Python (my new Love)
    3. Programming 4 Security and Incident Response

While this list is not comprehensive, it does reflect my current approach and goals for this blog.  Please be patient as I work on building this site back up in my spare time.

I’m open to any requests or suggestions to the above list.

 

Web and Tech Guy…