|
|
|
Welcome to the Basic Analysis and Security Engine (BASE) project
|
|
|
|
|
|
Jun 01 19:04:55 Bascially, I'll talk for a minute, hand it over to Kevin.. and he'll get it underway.
Jun 01 19:05:26 I just wanted to say thanks for coming, we really appreciate the support we've received from the community.
Jun 01 19:05:55 Our project was met with a bit of resistance at first, I think because people were set in their ways...
Jun 01 19:06:20 ... but now that we have a good project with signifcant development taking place, I'm glad to see that people are embracing the project..
Jun 01 19:06:48 I know it's not a huge number..
Jun 01 19:07:05 but we'll break the 17,000 download mark tonight...
Jun 01 19:07:15 So, thanks, again...
Jun 01 19:07:25 Kevin... you are the man..
Jun 01 19:07:29 ok...
Jun 01 19:07:37 and the crowd roars
Jun 01 19:07:39 First I would like to also thank everyone for coming....
Jun 01 19:07:51 and ask you to forgive my horrible spelling....
Jun 01 19:08:32 This project has grown quite a bit in the time we have been here, much faster then I ever thought possible.
Jun 01 19:08:56 I also believe that it has quite a ways to go to become the real "security engine" I believe it can be.
Jun 01 19:09:31 1.1.3 is about to be released
Jun 01 19:09:38 and 2.x is still being designed.
Jun 01 19:10:09 so we felt that this was the perfect time, to ensure that our ideas for the new version meets with our abilities and our users needs
Jun 01 19:10:13 hence this meeting.
Jun 01 19:10:16 ...
Jun 01 19:10:33 1.x is based, as everyone knows on the original ACID source code.
Jun 01 19:10:46 and while that code works and meets our needs at this point.
Jun 01 19:10:59 most of the features that are being requested and bugs that need to be fixed
Jun 01 19:11:09 are harder or impossible to do with this codebase.
Jun 01 19:11:32 because of this, we are hoping to create a modular system with 2.x
Jun 01 19:11:51 while not losing the simplicity of install and simple power that is the main reason people use BASE.
Jun 01 19:12:03 ....
Jun 01 19:12:14 So, unless something comes up that changes this,
Jun 01 19:12:34 2.x will be the main focus of development from the release of 1.1.3
Jun 01 19:12:51 now, we will be fixing bugs that can be and even implementing simple RFE's to 1.x
Jun 01 19:13:06 but if we are going to be successful with the next version, we must focus on it
Jun 01 19:13:13 if only because our resources are limited.
Jun 01 19:13:24 (we are all volunteers!)
Jun 01 19:13:34 ...
Jun 01 19:13:48 2.x will be a system that breaks apart the GUI and the logic.
Jun 01 19:14:03 the intertwining of those two pieces has caused most of the headaches in 1.x
Jun 01 19:14:36 The logic system will run on the web server as a series of PHP scripts.
Jun 01 19:15:00 these scripts will accept requests and send back streams of data.
Jun 01 19:15:15 The GUI will then be able to determine the best way to display these streams.
Jun 01 19:15:38 The primary interface will be a web client that will work similar to 1.x
Jun 01 19:16:38 the web client will then just POST to the web logic system to get responses.
Jun 01 19:16:57 We will then be able to support various front ends.
Jun 01 19:17:15 Thick clients, Dashboard type widgets, and other applications.
Jun 01 19:18:02 this will enable other applications to use the logic system of BASE to retrieve data.
Jun 01 19:18:27 Right now other apps, such as OSSIM, have to package BASE as a part of their system and work around the tie-ins.
Jun 01 19:18:33 ....
Jun 01 19:18:56 We will also be able to create various visibility systems using this.
Jun 01 19:19:36 By this I mean, when we authenticate a user, the GUI will be able to determine what they see, instead of the logic being mixed in there...
Jun 01 19:20:07 It will also let us take advantage of the power of the various systems we build on.
Jun 01 19:20:21 ....
Jun 01 19:20:38 Another major feature will be the improved reporting system.
Jun 01 19:21:10 We would like the system to generate reports that can be used for everything from day-to-day information, to management level "pretty graphs"
Jun 01 19:21:38 ....
Jun 01 19:22:08 We will also be building what I believe Joel is calling an "Incident Response Workflow"
Jun 01 19:22:22 :D
Jun 01 19:22:40 This will allow people to respond to alerts and work amongst many handlers within the BASE system
Jun 01 19:23:36 We have had many people ask for this type of feature and within the 1.x code train, it doesn't seem feasible.
Jun 01 19:24:41 We are hoping that this basic idea babble will be grown through what users need and what people are willing to code.
Jun 01 19:24:42 ....
Jun 01 19:24:52 Which brings us to the what I think we need....
Jun 01 19:25:14 First, we need to figure out a system to better test releases.
Jun 01 19:25:35 Chrisitan has done a great job testing what ever we throw at him, but he is only one guy.
Jun 01 19:25:49 and I am the first to admit, I commit code that needs to be backed out!
Jun 01 19:26:10 I would also like to grow the development team, with people who really want to code.
Jun 01 19:26:29 We also need to create better documentation.
Jun 01 19:26:52 The install guides by Patrick Harper and others are great.
Jun 01 19:27:13 But if we are going to build a workflow system, we will need to create in-system help....
Jun 01 19:27:45 Kevin, I actually had a good friend of mine, and known snort contributor to the community, volunteer to help us out in the documentation arena today. I'll be adding him to the team soon...
Jun 01 19:28:00 And of course, I want to figure out how we can work closer with snort to ensure that as releases are coming out, we are ready for them...
Jun 01 19:28:15 So far, I believe that has been hampered by me not upgrading
Jun 01 19:28:28 .....
Jun 01 19:29:04 Well I have babbled for almost half an hour so I would like to open it up to questions.... First, Joel have you gotten any that we can start with?
Jun 01 19:29:22 hhoffman posed a good question..
Jun 01 19:29:40 I can repose (i already have another though) :-)
Jun 01 19:29:44
Jun 01 19:29:46 Is base going to turn into a SIM?
Jun 01 19:29:58 Security Information Manager.
Jun 01 19:30:11 I cant answer that...
Jun 01 19:30:14 basically because
Jun 01 19:30:22 I believe that SIM is a marketing term
Jun 01 19:30:39 We are going to become a system to handle alerts.
Jun 01 19:30:44 well, what I'm curious about the the features normally associated with a SIM
Jun 01 19:30:49 I believe that in order for an analyst to get the most out of a tool, and to keep the analyst from working with 5 different tools to get his information, he should only have to look at one piece.
Jun 01 19:30:56 snmp traps, syslog, flow data, etc.
Jun 01 19:30:56 I agree..
Jun 01 19:31:12 Right now, we are looking at reporting the data out of BASE.
Jun 01 19:31:25 That is one of the reasons we are splitting the logic and the UI.
Jun 01 19:31:35 that leads to my other question...
Jun 01 19:31:41 but I am not sure the right place for alerting is within BASE when Snort does it well...
Jun 01 19:31:53 are there any plans to work with the snort team to modify the db schema at all?
Jun 01 19:32:11 There have been a number of ideas around the snort db and the BASE db...
Jun 01 19:32:26 not sure if we are able to change the db......
Jun 01 19:32:38 problem is, if we work with Sourcefire to change the db, it would break countless other tools.
Jun 01 19:32:50 hmm, good point
Jun 01 19:33:25 we'd like to build more tables inside the db
Jun 01 19:33:42 or a seperate "along side" db in order to perform the queries.
Jun 01 19:33:47 And we will be creating a Barnyard plugin for any changes we make to our tables....
Jun 01 19:34:07 yeah, that's what I'm currently resorting to with our homebrew reporting system
Jun 01 19:34:17 oh! that's good to know
Jun 01 19:35:27 We had a question about our statement: "the statement "when we authenticate a user, the GUI will be able to determine what they see" [/do?]. This implies you can bypass authentication by bypassing the GUI, and going straight to web requests. Or does it?
Jun 01 19:35:34 (Question was from cisc0man)
Jun 01 19:35:44 The short answer is no.
Jun 01 19:35:46 no it doesn't...
Jun 01 19:35:53 You will not be able to bypass user authentication.
Jun 01 19:36:00 We are going to be building an authentication system that will be pluggable...
Jun 01 19:36:02 Think PAM
Jun 01 19:36:04 (unless you completely shut it off)
Jun 01 19:36:36 it will happen at the logic portion but the GUI portion will also tie into it...
Jun 01 19:36:50 where display of menus and such are there....
Jun 01 19:37:18 OK, that sounds better :-) Thanks.
Jun 01 19:37:26 ;>
Jun 01 19:37:29 Another question: have you guys thought about using the horde framework at all?
Jun 01 19:37:40 I have never really looked at the Horde framework..
Jun 01 19:37:44 (Question was from hhoffman)
Jun 01 19:38:14 I have heard good things about it though... so we probably should at least look into it...
Jun 01 19:38:22 it's certainly something we can look into.
Jun 01 19:38:23 it's quite nice... the framework is there for many things and then ppl write modules to interact with it
Jun 01 19:39:04 cool.
Jun 01 19:39:19 kevin, can we talk about a couple ideas, and see what people think about them?
Jun 01 19:39:29 sure...
Jun 01 19:39:50 Talk about our Incident Grouping Workflow.
Jun 01 19:39:54 for a sec...
Jun 01 19:40:14 What we want is the ability to be able to assign events to an "Incident Number"
Jun 01 19:40:24 now, you guys are thinking "Alert Groups"
Jun 01 19:40:27
Jun 01 19:40:32 which is something we already have, and no one uses.
Jun 01 19:40:42 But we want to take it a step further.
Jun 01 19:41:08 We want to be able to assign certain events/priorities/sensors to a person
Jun 01 19:41:17 and have them work it from "cradle to grave"
Jun 01 19:41:53 This will help a lot in the large systems or edu's where various groups are responsible for sections of the network or applications
Jun 01 19:41:55 one of the present limitations with this is, that when an event is deleted or Archived, it's not in the Incident (or alert group) anymore.
Jun 01 19:42:04 we're going to have to move these to a seperate table.
Jun 01 19:42:13 any thoughts about allowing hooks into comerical tools like Remedy or Service Center?
Jun 01 19:42:30 yes....
Jun 01 19:42:51 It would just be another front end interface.
Jun 01 19:43:05 under an incident group..
Jun 01 19:43:23 and reconstruct data..
Jun 01 19:43:40 attach various word documents or excel spreadsheets to the Incident Group..
Jun 01 19:43:47 and have them flow..
Jun 01 19:44:30 what else would you guys like to see as part of that?
Jun 01 19:44:34 I'm most interested in separating "new, virgin" alerts from those that have been [deleted]/archived/alert-grouped/workflowed.
Jun 01 19:44:46 that would be key in making it work
Jun 01 19:45:05 what about a real-time alert console?
Jun 01 19:45:18 as opposed to an auto-refresh method..
Jun 01 19:45:38 I think on front ends that can handle it, definitely.
Jun 01 19:45:43 The alert group incidents still appear in the pool of current alerts, would like them to "go away" from most recent, today, etc.
Jun 01 19:45:46 are you also thinking about escalation management? (ie -> analyst hasn't done anything escalate to manager)
Jun 01 19:45:57 But still be available for "any sig, src or dst" for example.
Jun 01 19:46:06 hhoffman, I consider that workflow.... but yes
Jun 01 19:46:32 ciscoman, that would depend on the settings of the person,
Jun 01 19:46:34 gotcha
Jun 01 19:46:59 for example, if you were on a "new" screen, they would only be on the un-flowed alerts
Jun 01 19:47:13 but if you were in a historical type mode, they would all appear...I think
Jun 01 19:47:39 that will be something that will take extensive testing with the community to make sure it's what they want.
Jun 01 19:48:15 That's the idea. Now you have to delete/archive to get them out of current, but for long-term things it's nice to search for anything you have available.
Jun 01 19:48:50 so you're talking about crossing databases for historical data? Am I reading that right?
Jun 01 19:49:05 (just trying to clarify)
Jun 01 19:49:35 I think we should move away from the archive and current db and more toward tables within the same db
Jun 01 19:49:47 i agree...
Jun 01 19:49:54 simplifiy reporting...
Jun 01 19:50:10 but it will require that we work toward using the power of each DB system to its fullest.
Jun 01 19:51:20 okay..
Jun 01 19:51:38 Another idea we had was the "session" playback theory.
Jun 01 19:51:54 a few people have asked for it.
Jun 01 19:52:00 As I am sure you are familiar, in the Snort rule options.. there is "tag"
Jun 01 19:52:14 which will record further sessions as it relates to an alert.
Jun 01 19:52:30 What we want to do is to be able to "aggregate" (is that the right word) this data
Jun 01 19:52:40 so with one click, we can reconstruct a "session"
Jun 01 19:52:54 thoughts?
Jun 01 19:53:01 this would depend on whether some tie in is logged to the DB by snort
Jun 01 19:53:08 I dont know if it is...
Jun 01 19:53:22 Providing "real-time" view or better some reaction would be great. But it isn't something a web interface can do. We might want to write a wrapper which gets the alerts from Snort, acts on them if required (sends an email/snmp trap) and stores them in the DB.
Jun 01 19:53:25 i am pretty sure it is..
Jun 01 19:54:02 Maybe someone can square me away, because I can't remember off the top of my head (too many other thoughts..)
Jun 01 19:54:07 I am not sure the std snort DB schema has a column for it, but tagged alerts contain ref info.
Jun 01 19:54:17 I think the sid's for the tag and the sid for the rule that caused the tag are the same.
Jun 01 19:54:31 Although they appear to be somewhat b0rked from my experience. (FYI)
Jun 01 19:54:35 can't be sids...
Jun 01 19:54:36 sorry, got distracted. I was hoping not to have to pull from multiple DBs, just what was in current alert DB. Perhaps a "Seen" flag. Now I'll hush.
Jun 01 19:54:52 dont hush...
Jun 01 19:54:57 It should all be in one DB...
Jun 01 19:55:07 are there any plans to tie in MAC->IP matching for local systems... this helps us with historical data in networks where we have short dhcp leases
Jun 01 19:55:20 hadn't thought about it...
Jun 01 19:55:30 but worth looking into I believe...
Jun 01 19:55:37 there is something that is cross-correlated between the signatures and the "tag"'s.. but I can't remember what field there is... Perhaps Matt scan clairfy on that one..
Jun 01 19:55:56 hhoffman, are you dhcp host names the same?
Jun 01 19:56:12 what if we built an option to be able to sort by host names?
Jun 01 19:56:56 *on that note.. I don't think Snort logs layer 2 data to db..someone correct me if I am wrong..*
Jun 01 19:57:08 eslerj: the host names stay the same
Jun 01 19:57:33 but we would need to capture the host name at time of the alert.
Jun 01 19:57:41 right.
Jun 01 19:57:42 right... but in our case we have a pcap app running that logs arp traffic into the db from starting/ending/last seen time
Jun 01 19:57:52 hhoffman, you're talking about something like arpwatch?
Jun 01 19:58:03 that can then be correlated to the time of the alert for that ip
Jun 01 19:58:20 zroone: yep, similiar to that but ours logs directly to the db
Jun 01 19:58:28 this would probably call for one of the stand alone scripts to read your db and coorrelate
Jun 01 19:58:28 what if we could make BASE read arpwatch logs?
Jun 01 19:58:53 or dhcpd?
Jun 01 19:59:02 that would be nice. but, arpwatch can't log to db?
Jun 01 19:59:03 * eslerj nods
Jun 01 19:59:25 arpwatch can't AFAIK
Jun 01 19:59:45 but we can create programs, if they dont exist, to read the logs and write to the DB...
Jun 01 19:59:56 yep
Jun 01 19:59:56 * eslerj says, "By the way... We just acheived a download milestone.. 17,006"
Jun 01 19:59:59 found it in google: http://www.thtech.net/article/12
Jun 01 20:00:18 admit it Joel, you are d'ling copy after copy
Jun 01 20:00:30 they've caught onto me.
Jun 01 20:00:31 yep, we used that as our base and then modified it
Jun 01 20:01:31 One point I forgot to make is that because of the expanse of features we want to implement and the limit on volunteers, we need to create everything in a modular fashion so that people dont have to wait 7 years for 2.0
Jun 01 20:02:01 eslerj: if you email me i'll write something up for how tag logs stuff.
Jun 01 20:02:10 yeah, that's a really good idea.
Jun 01 20:02:11 thanks
Jun 01 20:02:18 thanks matt..
Jun 01 20:02:31 I know I know.. i just can't remember..
Jun 01 20:02:38 Idea #3...
Jun 01 20:03:02 BASE's ability to read other IDS's.
Jun 01 20:03:23 how about nessus results?
Jun 01 20:03:25 or perhaps translate other IDS db schemas to a common schema.
Jun 01 20:03:41 hhoffman, hey.. this isn't site protector.. ;)
Jun 01 20:03:44 I would want to do the second... translation is easier....
Jun 01 20:03:52 what about IDMEF? how many IDSs implement this?
Jun 01 20:03:53 and can be done with stand alone modules in perl.
Jun 01 20:04:27 | | | | | |