Thursday, 2012-06-07

*** jgriffith is now known as jgriffith_away00:00
*** jakedahn is now known as jakedahn_zz00:09
*** johnpostlethwait has quit IRC00:14
*** arosen has joined #openstack-meeting00:15
*** matwood has joined #openstack-meeting00:15
*** ryanpetr_ has joined #openstack-meeting00:23
*** jeblair has quit IRC00:23
*** jeblair has joined #openstack-meeting00:24
*** ryanpetrello has quit IRC00:27
*** nati_uen_ has quit IRC00:34
*** troytoman-away is now known as troytoman00:42
*** matwood has quit IRC00:47
*** joearnold has joined #openstack-meeting00:47
*** nati_ueno has joined #openstack-meeting01:00
*** joearnold has quit IRC01:01
*** Mandell has quit IRC01:03
*** joearnold has joined #openstack-meeting01:06
*** markmcclain has quit IRC01:06
*** markmcclain1 has joined #openstack-meeting01:06
*** joearnold has quit IRC01:07
*** s0mik has quit IRC01:12
*** lloydde has joined #openstack-meeting01:19
*** s0mik has joined #openstack-meeting01:22
*** s0mik has quit IRC01:29
*** gyee has quit IRC01:35
*** s0mik has joined #openstack-meeting01:38
*** shang_ has joined #openstack-meeting01:47
*** shang_ has quit IRC01:47
*** danwent has quit IRC01:48
*** s0mik has quit IRC01:50
*** lloydde has quit IRC01:50
*** markmcclain1 has quit IRC01:54
*** shang has quit IRC01:55
*** shang has joined #openstack-meeting01:59
*** PotHix has quit IRC02:00
*** ryanpetr_ has quit IRC02:08
*** matwood has joined #openstack-meeting02:18
*** edygarcia has joined #openstack-meeting02:25
*** kiffer84 has quit IRC02:26
*** kiffer84 has joined #openstack-meeting02:28
*** ywu has quit IRC02:29
*** lloydde has joined #openstack-meeting02:46
*** bencherian has quit IRC02:55
*** adjohn has quit IRC02:56
*** anderstj has joined #openstack-meeting02:59
*** nati_ueno has quit IRC03:04
*** anderstj has quit IRC03:15
*** edygarcia has quit IRC03:19
*** anderstj has joined #openstack-meeting03:25
*** markvoelker1 has quit IRC03:30
*** littleidea has joined #openstack-meeting03:34
*** anderstj has quit IRC03:39
*** lloydde has quit IRC03:40
*** edygarcia has joined #openstack-meeting03:42
*** anderstj has joined #openstack-meeting03:45
*** jakedahn_zz is now known as jakedahn03:48
*** dwcramer has quit IRC03:51
*** troytoman is now known as troytoman-away03:54
*** edygarcia has quit IRC03:58
*** edygarcia has joined #openstack-meeting03:58
*** novas0x2a|laptop has quit IRC03:59
*** edygarcia has quit IRC04:11
*** jakedahn is now known as jakedahn_zz04:28
*** garyk has quit IRC04:31
*** anderstj has quit IRC04:32
*** milner_ has quit IRC04:46
*** Mandell has joined #openstack-meeting04:51
*** milner_ has joined #openstack-meeting04:58
*** gyee has joined #openstack-meeting04:59
*** bencherian has joined #openstack-meeting05:02
*** novas0x2a|laptop has joined #openstack-meeting05:04
*** shang has quit IRC05:05
*** garyk has joined #openstack-meeting05:51
*** anderstj has joined #openstack-meeting06:00
*** Mandell has quit IRC06:03
*** gakott has joined #openstack-meeting06:03
*** garyk has quit IRC06:04
*** gakott has quit IRC06:04
*** garyk has joined #openstack-meeting06:05
*** littleidea has quit IRC06:16
*** nati_ueno has joined #openstack-meeting06:22
*** anderstj has quit IRC06:24
*** dayou has joined #openstack-meeting06:24
*** shang has joined #openstack-meeting06:35
*** nati_ueno has quit IRC06:41
*** nati_ueno has joined #openstack-meeting06:45
*** gyee has quit IRC06:51
*** bencherian has quit IRC06:55
*** nati_ueno has quit IRC07:00
*** nati_ueno has joined #openstack-meeting07:00
*** nati_ueno has quit IRC07:11
*** shang has quit IRC07:16
*** nati_ueno has joined #openstack-meeting07:17
*** ttrifonov_zZzz is now known as ttrifonov07:20
*** mnewby has quit IRC07:33
*** nati_uen_ has joined #openstack-meeting07:37
*** nati_ueno has quit IRC07:39
*** ehm has quit IRC07:42
*** derekh has joined #openstack-meeting08:19
*** nati_uen_ has quit IRC08:59
*** sandywalsh has quit IRC09:01
*** shang has joined #openstack-meeting09:35
*** shang has quit IRC10:31
*** shang has joined #openstack-meeting10:47
*** ijw has joined #openstack-meeting11:08
*** ayoung has quit IRC12:13
*** dprince has joined #openstack-meeting12:32
*** sandywalsh has joined #openstack-meeting13:14
*** dprince has quit IRC13:14
*** ayoung has joined #openstack-meeting13:15
*** jgriffith_away is now known as jgriffith13:22
*** dolphm has joined #openstack-meeting13:36
*** mnaser has joined #openstack-meeting13:36
*** dolphm_ has joined #openstack-meeting13:36
*** rnirmal has joined #openstack-meeting13:36
*** dolphm has quit IRC13:40
*** thatsdone has joined #openstack-meeting13:42
*** anderstj has joined #openstack-meeting13:44
*** edygarcia has joined #openstack-meeting13:50
*** joesavak has joined #openstack-meeting13:53
*** thatsdone has quit IRC13:57
*** mnaser has quit IRC14:05
*** tongli|2 has joined #openstack-meeting14:05
*** mnaser has joined #openstack-meeting14:06
*** bencherian has joined #openstack-meeting14:18
*** nikhil_ has joined #openstack-meeting14:26
*** nikhil_ is now known as Guest5273814:26
*** krtaylor has quit IRC14:26
*** tongli|2 has quit IRC14:28
*** tongli has joined #openstack-meeting14:28
*** danwent has joined #openstack-meeting14:33
*** mattray has joined #openstack-meeting14:34
*** tongli has quit IRC14:39
*** tongli has joined #openstack-meeting14:39
*** bencherian has quit IRC14:44
*** adjohn has joined #openstack-meeting14:52
*** adjohn has quit IRC14:52
*** Mandell has joined #openstack-meeting14:55
*** dwcramer has joined #openstack-meeting14:55
*** krtaylor has joined #openstack-meeting14:56
*** littleidea has joined #openstack-meeting14:58
*** mnewby has joined #openstack-meeting15:04
*** rash has joined #openstack-meeting15:06
*** anderstj has quit IRC15:07
*** dcramer_ has joined #openstack-meeting15:11
*** dwcramer has quit IRC15:12
*** markmcclain has joined #openstack-meeting15:14
*** GheRivero has quit IRC15:16
*** sleepsonthefloor is now known as sleepsonzzz15:23
*** Mandell has quit IRC15:29
*** GheRivero has joined #openstack-meeting15:30
*** edygarcia_ has joined #openstack-meeting15:32
*** bencherian has joined #openstack-meeting15:32
*** edygarcia has quit IRC15:35
*** edygarcia_ is now known as edygarcia15:35
*** blamar has joined #openstack-meeting15:38
*** ttrifonov is now known as ttrifonov_zZzz15:41
*** garyk has quit IRC15:45
*** ttrifonov_zZzz is now known as ttrifonov15:46
*** edygarcia_ has joined #openstack-meeting15:47
*** edygarcia has quit IRC15:50
*** edygarcia_ is now known as edygarcia15:50
*** ewindisch has joined #openstack-meeting15:53
*** dayou has quit IRC15:57
*** dachary has joined #openstack-meeting15:57
nijabao/15:58
dachary\o15:59
jd___hi15:59
dhellmannhi16:00
jd___let's go then :)16:00
jd___#startmeeting16:01
jd___#meetingname ceilometer16:01
openstackMeeting started Thu Jun  7 16:01:02 2012 UTC.  The chair is jd___. Information about MeetBot at http://wiki.debian.org/MeetBot.16:01
jd___#link https://lists.launchpad.net/openstack/msg12851.html16:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:01
openstackThe meeting name has been set to 'ceilometer'16:01
jd___#chair nijaba dachary16:01
openstackCurrent chairs: dachary jd___ nijaba16:01
jd___#topic actions from previous meetings16:01
*** openstack changes topic to "actions from previous meetings"16:01
*** dcramer_ has quit IRC16:01
jd___dhellmann: so how was your demo? :)16:01
*** edygarcia_ has joined #openstack-meeting16:01
dhellmann:-)16:02
dhellmannit went well16:02
*** matwood has quit IRC16:02
dhellmannI was able to show messages passing from the agent to the collector and being logged, both from notifications and polling16:02
dacharydhellmann: congratulations16:02
dhellmannthank you all, again, for all of the work you've put in. I wouldn't be nearly this far if I was doing it all myself.16:03
nijabadhellmann: impressive!16:03
*** edygarcia has quit IRC16:04
*** edygarcia_ is now known as edygarcia16:04
jd___great dhellmann :)16:04
jd___nijaba: anything to report since you had a couple of action items?16:04
nijabaWell, I commented on the bug and I started the thread on configuration handling16:05
jd___fair enough :)16:05
jd___#topic Storage backend (high availability, SPOF etc.)16:06
*** openstack changes topic to "Storage backend (high availability, SPOF etc.)"16:06
*** matwood has joined #openstack-meeting16:06
jd___now it's time to discuss how to store the collected data :)16:07
dhellmannour ops team is recommending postgres, in part because of familiarity and in part because they think it will handle the scale16:07
jd___dhellmann wrote something cf https://lists.launchpad.net/openstack/msg12884.html16:07
jd___#link https://lists.launchpad.net/openstack/msg12884.html16:07
dhellmannthey have specifically warned me off of mongodb16:07
dhellmannbut I don't expect everyone to want to use the same tool, which is why I proposed a plugin api16:08
dhellmann(thanks, jd___)16:08
jd___dhellmann: I agree with your ops, but I'd suggest to use sqlalchemy as an abstract just because OS already did this choice16:08
dhellmannyes, absolutely16:08
*** ijw has quit IRC16:08
dhellmannI intend to have an "sql" or "rdbms" plugin that uses sqlalchemy16:08
jd___but as you stated, this can be pluggable in our case, as you proved and wrote so we can start with one plugin16:08
*** ijw has joined #openstack-meeting16:08
dhellmannit may be simplest to create a mongo plugin for testing and experimentation, but we wouldn't use it in production at dreamhost16:09
nijabaplease, do NOT use SQL Alchemy.  That would prevent us from using any noSQL db.  You may want to use is in one of the pluging, but not as the plugin method16:09
jd___nijaba: it's not the plugin method16:09
nijabajd___: then that's fine16:09
jd___:)16:09
* nijaba thinks that's one of the bad elements remaining in OpenStack at the moment16:10
jd___dhellmann: well if we both want to use SQL I think so it's likely we can work on this plugin first16:10
dhellmannright, there would be a single plugin with a name like "rdbms" that uses sqlalchemy to talk to your DB of choice, but the plugin API would be a higher level thing16:10
*** chrisfer has joined #openstack-meeting16:10
dhellmannagreed, jd___16:10
nijabathat sounds good to me16:11
dhellmannthe plugin will need more methods so the API server can use it to query the database, too, but I haven't given those any thought16:11
dhellmannI would expect them to map pretty closely to the queries in the API itself, though16:11
jd___#agreed jd and dhellmann to focus on an SQL plugin storage16:11
*** garyk has joined #openstack-meeting16:11
nijabawell, looks like we only have have the storage part so far16:11
jd___dhellmann: indeed16:11
nijabawe need to define the maping to the API we defined earlier16:11
dhellmannjd___, do you know anything about how the other OS components handle database migrations?16:12
jd___dhellmann: I know they use 'migrate'16:12
jd___I even wrote one or two migration stuff in the last months16:12
dhellmannright, nijaba. We should at least add a method to retrieve the raw data so we can test getting data in and out16:12
*** joearnold has joined #openstack-meeting16:12
dhellmannjd___: oh, good, so you can do that part! :-)16:13
*** s0mik has joined #openstack-meeting16:13
nijabadhellmann: that would sound like a nice first step16:13
jd___dhellmann: it will only be needed for version 2! :-P16:13
dhellmannwell, we have to have something to initialize the database, right?16:13
jd___right, but sqlalchemy does that for us AFAIK16:13
*** ewindisch has quit IRC16:14
dhellmannI know it can be used to create the schema, but I think you have to tell it to do that explicitly16:14
dhellmannwe can take that part of the discussion to the mailing list, though16:14
jd___otherwise OS uses this to upgrade between releases: http://code.google.com/p/sqlalchemy-migrate/16:14
jd___IIRC16:14
jd___dhellmann: sure :)16:14
nijabaso the plugin should abstract a migrate function...16:15
jd___so everybody agrees on the plugin system proposed by dhellmann ?16:15
nijabait's a good start, I think16:15
dhellmann+116:15
jd___nijaba: if it's needed, each plugin handles its migration; i think mongodb migration can be easy to do since you don't have to do anything to add fields ;)16:15
dachary+116:15
jd___#agreed use the plugin system proposed by dhellmann at https://lists.launchpad.net/openstack/msg12884.html16:16
dhellmann#action dhellmann: submit plugin branch for review and merging16:16
nijabaas long as we agree to always go through the abstraction to talk to the db, I think we are fine16:16
dhellmannnijaba, agreed16:16
jd___sure16:17
dacharydhellmann: could you point me to the abstraction related to how the plugin is used to query the database ?16:18
dhellmanndachary, there aren't any query methods, yet16:18
*** dcramer_ has joined #openstack-meeting16:18
dhellmannjust like there's a method to store a new event, there would be one or more methods to ask for event data16:18
dhellmannone would ask for all of the raw events, filtered by account, user, etc -- whatever the API args are16:18
nijabain fac, I think we will have one method per API call type16:19
dacharythat's non trivial to abstract. Or do you have an abstract model in mind already ?16:19
dhellmannyeah, probably16:19
dhellmannno, I haven't gotten that far16:19
nijabathe API is the abstraction...16:19
dacharynijaba: then the database plugin will be in charge of interpreting the API calls. That works for me.16:19
dhellmannwe may be able to build the API server using fewer plugin API calls (too many APIs…) but they will map closely16:20
*** sleepsonzzz is now known as sleepsonthefloor16:20
nijabaagreed16:20
dhellmannthe API service will do some parameter validation, call the plugin to get the data, then format it for return16:20
nijabasounds like MVC applied to DB...16:20
dhellmannsomething like that :-)16:20
nijabajd___: what do you think?16:21
jd___I think like dhellmann :)16:22
nijabajd___: should we capture on action on building a few example API calls to the plugin?16:22
jd___we restrict the use of a storage plugin to one?16:22
nijabajd___: one at a time? yes16:23
jd___nijaba: not sure it's that useful since we have nothing (no code) related to API for now16:23
dhellmannjd___, yes, each collector instance would be using only one storage plugin but you could have multiple clusters writing the data to different databases if you wanted16:23
nijabaso maybe we should call it an Engine rather than a plugin16:23
jd___yeah I meant one at time at runtime :)16:23
jd___nijaba: +116:23
nijabalet's learn from quantum's mistake here ;)16:24
jd___lol16:24
* dhellmann shakes head16:24
dhellmann"engine" it is16:24
jd___#action dhellmann rename plugin to engine for storage backend ex-plugin-now-engine system16:24
jd___I hope that's clear16:25
dhellmannI will do that before submitting the code for review16:25
nijabathanks dhellmann16:25
dacharyhow do we address SPOF ?16:25
dhellmannwith the database?16:26
nijabawell, that's why I wanted us to be able to suport NOSQL dbs16:26
dhellmannhow does nosql relate to spof?16:26
dacharydhellmann: you mean by using a postgresql setup with no spof ?16:26
dacharyfor instance16:26
jd___there's HA in SQL too16:26
nijabatrue, but it is a bit easir to setup multiple conf=current master with some NoSQL than with postgres16:27
dhellmannI'm not an ops expert, but yes, my ops team didn't seem concerned about postgresql as a SPOF so I assume they are planning to cluster it16:27
dhellmannnijaba, that can be true16:27
jd___good ops :)16:28
dhellmannthe feedback I was getting was that mongo might fall over if we push too much data in16:28
dhellmannthat's anecdotal, but I have to trust my ops team, don't I?16:28
dacharyin practice when you need to follow a method to implement "no SPOF", it usualy does not happen.16:28
* jd___ thinks it's a religious war we don't want to get into16:28
nijabadhellmann: that's not the view from our ops here, but used to be until 6 month ago16:28
* nijaba agrees with jd___16:28
jd___(flat file anyone?)16:29
dhellmannpickle ftw16:29
dacharyI think the idea of introducting a "no SPOF" in the definition of the database was to make it a default instead of a possibility that needs to be implemented on top.16:29
jd___I think it was nijaba wanting to push nosql :)16:29
dhellmanndo any of the SPOF solutions for databases require application code changes?16:29
nijabadachary: how would you do this?  Write to 2 dbs at once?16:29
nijabamy point was: let's not get stuck with an SQL only impelmentation.  If we have an abstraction layer, then we can let the community play around and come up with solutions16:31
dhellmannI don't think we want the app to be responsible for database reliability. All of the "real" solutions I've seen support some sort of clustering16:31
dhellmannexactly, we can push that responsibility down into the plugin and not worry about it in the core app16:31
dacharyI'd use mongodb because it has this concern built in from the start. Otherwise i'd follow Florian Haas advices regarding HA ;-)16:31
dhellmannhas anyone done any work to estimate the amount of data they will be generating?16:32
nijabadachary: I am hopping to get some resources soon to work on a mongodb engine...16:32
dacharynijaba: great ;-)16:32
dacharyI'm just stating a concern but I don't see this as a blocker.16:32
nijabadhellmann: I can take the action to build a calculator16:33
dhellmannnijaba, excellent, that would be a real help16:33
* nijaba think about a google spreadsheet, if that suits everyone16:33
*** nati_ueno has joined #openstack-meeting16:33
dhellmannI have some estimates for the number of VMs we expect to have, but have not had time to do the math on data size, yet16:33
jd___np16:33
jd___think about Swift too16:34
nijabaswift as a source for metering message, or as a storgae engine?16:34
jd___I meant source for metering16:34
nijabak16:34
*** Ravikumar_hp has joined #openstack-meeting16:34
nijaba#action nijaba to propose a google spreadsheet calculator to estimate volume of metering message (including nova, swift, cinder, quantum)16:36
jd___anything else?16:37
dhellmannI think that covers everything I had related to storage16:37
dacharywas there a agree on the fact that the database engine has a function to interpret the API queries in addition to the function to store the data ?16:37
dacharya agree => a "dash agree" ;-)16:38
nijabaI think there was.16:38
dhellmanndachary, I think we agreed there would likely be several methods related to querying16:38
dhellmannand that we still need to define them16:38
dacharyI dont see the dash agree matching this16:39
dhellmannoh, we may not have recorded it that way16:39
dhellmannI just meant we seemed to come to consensus :-)16:39
dacharyabsolutely, I got that too ;-)16:39
dhellmannok16:39
jd___#agreed a database engine has a function to interpret the API queries in addition to the function to store the data16:40
dacharyjd___: thanks16:40
dhellmann#action dhellmann: start mapping API queries to database engine methods16:40
dhellmannI'll put together a wiki page with some proposals and we can discuss on the list16:41
nijabadhellmann: if you prime me with a first example, I can take care of the declinations16:41
dhellmannnijaba, sounds good16:41
*** ryanpetrello has joined #openstack-meeting16:42
dhellmannwas there something else on the agenda for today?16:43
nijabaAgent configuration mechanism?16:43
jd___yep16:43
jd___moving on then16:43
jd___#topic Agent configuration mechanism16:43
*** openstack changes topic to "Agent configuration mechanism"16:43
dhellmanndid we agree on the list that we would use text config files and leave it up to ops to manage them, as with the other components?16:43
jd___that's my point of view at least16:44
nijabaso, I must say that I would not be happy with this for meter configuration16:44
nijabaI am fine with this being used for the global agent config16:44
dacharyI think there are merits on complementing the configurations mechanisms.16:44
nijababt I think we risk to have unuseable date captured if not all meter for a given value are set to report the same way, or report at all16:45
dacharyConfiguration engines like puppet or chef have limitations.16:45
dhellmannnijaba, I think I understand, but how often do you see the agent/meter configurations changing?16:45
nijabait is a real data consistency problem16:45
*** anderstj has joined #openstack-meeting16:45
nijabadhellmann: as often as marketing will ask for tsomthing new ;)16:45
dhellmannin that case, wouldn't it make sense to just collect as much data as possible?16:46
nijabathat would be the brute force approach16:46
dhellmannwell, yeah :-)16:47
dacharyI've worked with puppet recently and synchronisation with nagios : it's not pretty. We would be *much* better of using direct connections with the nagios plugins, if it was possible. Instead of going thru the puppet database.16:47
jd___I don't even see what could be configurable and go wrong with plugins for now16:48
nijabahow hard do you think it is to have the meter configuration stored and retrived by the agents?16:48
jd___except time sync problem but that we won't solve :)16:48
dhellmanntime sync? isn't that solved by ntp?16:48
nijabajd___: imagine that you want to capture cpu, but for some reason, only half of your host get that16:48
jd___dhellmann: I hope so :)16:48
nijabadhellmann: I think he meant frequency16:48
dhellmannah16:49
dhellmannnijaba, how would that happen? the ops configuration management tool should detect that a config is out of date and fix it, no?16:49
nijabaso my proposal is: agent are configured through trditinal means, butagent get meter config from the central collector16:50
nijabadhellmann: in theory, yes, but practice has shown this to not always be so true16:50
dacharyvery much in a same way a mysql database has communication with its slave for internal purposes, event when it's configured at install time using puppet16:51
nijabaand since this causes a real data consistency issue, this is why I am a bit pushy here16:51
dacharydhellmann: I would not trust puppet or chef to handle every use case16:51
dhellmanndachary, OK. Well, I trust my ops team to figure out how to make that work, but let's assume we need to have this feature for now and discuss what it might look like.16:52
nijabadhellmann: ask your ops if they trust puppet to set up a drbd cluster...16:52
dhellmannThe proposed API seemed more complicated than necessary.16:52
dhellmannwe use chef, but OK :-)16:52
dacharynijaba: another good example, yes.16:53
*** derekh has quit IRC16:53
nijabadhellmann: I am very open to changes16:53
*** rash has quit IRC16:53
dhellmannI propose a 2 step system.16:53
dhellmannOn startup, the agent "checks in" with the collector to retrieve its configuration16:53
dhellmannAt any other time, when the configuration is changed, the collector sends the new configuration to the agent. The agent discards its existing configuration and replaces it with the new settings.16:54
dacharywarning : 6 minutes left ;-)16:54
dhellmannwe probably need to move this discussion back to the list, then16:54
nijabadhellmann: do we use a cast in that case, or directed message (which implies maintaining a list of gents?)16:54
*** jakedahn_zz is now known as jakedahn16:54
nijabas/gents/agents/16:55
dhellmannnijaba, cast (assuming all agents are configured the same way)16:55
*** JoseSwiftQE has joined #openstack-meeting16:55
dacharyI propose we move to a vote on the principle and move the discussion on the implementation to the list.16:55
dhellmannok16:55
nijabadhellmann: that sounds good.  Do you want me to rework my proposal, or do you want to have a stab at it?16:55
dhellmannnijaba, I don't expect to use this feature so maybe you should do it? :-)16:55
nijabadhellmann: k, fair enough16:56
dhellmanndachary, we should also discuss/vote on whether this is a Folsom feature or a G feature16:56
nijabadhellmann: time box: feature which don't make it for a release are pushed to the next one....16:57
dacharydhellmann: is there a blocker for it to be a Folsom if someone works on it ?16:57
dacharyah16:57
dacharytimebox of course ;-)16:57
dhellmannI wouldn't block someone else from working on it, but I don't think it should have a high priority given all of the other things we have to do for Folsom16:58
*** jdurgin has joined #openstack-meeting16:58
dacharymakes perfect sense to me16:58
dhellmannI would rather see people working on pollsters and notification collection16:58
nijabadhellmann: here you find me in agreement16:58
dacharywho is in agreement to the proposal that agent are configured through traditional means, but agent get meter config from the central collector ?16:58
* nijaba really hopes to be able to put some effort where is mouth is16:59
dacharynijaba: you'll have to ;-)16:59
nijaba+116:59
dachary+116:59
dhellmann-116:59
jd___016:59
dachary#agreed agent are configured through traditional means, but agent get meter config from the central collector ?17:00
dacharynijaba: you take action ?17:00
*** dcramer_ has quit IRC17:00
nijabayes, my action is to rework my proposal on the basis of what was proposed by dhellmann17:00
dacharynijaba: dash action please ;-)17:00
*** Carlos_Swift has joined #openstack-meeting17:01
nijaba#action nijaba to rework meter configuration proposal on the basis of discussion17:01
jd___end time guys17:01
*** Sam___ has joined #openstack-meeting17:01
dacharyyes17:01
nijabathanks *17:01
jd___I can hear dhellmann' stomach17:01
dacharytnaks17:01
jd___#endmeeting17:01
dhellmannplease go look at the open code reviews!17:01
nijabatnaks too17:01
*** openstack changes topic to "OpenStack meeting channel. See http://wiki.openstack.org/Meetings for schedule and http://eavesdrop.openstack.org/meetings/openstack-meeting/ for meeting logs"17:01
openstackMeeting ended Thu Jun  7 17:01:52 2012 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-06-07-16.01.html17:01
*** Sam___ has quit IRC17:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-06-07-16.01.txt17:01
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-06-07-16.01.log.html17:01
dhellmannand thanks, everyone17:02
nijabadhellmann: on my way!17:02
dacharydhellmann: I'm on it17:02
*** Sam_swift_Racksp has joined #openstack-meeting17:02
*** donaldngo_hp has joined #openstack-meeting17:03
*** dendro-afk is now known as dendrobates17:03
*** bencherian has quit IRC17:06
*** Carlos_Swift has quit IRC17:06
*** bencherian has joined #openstack-meeting17:07
davidkranzI don't see Jay or Daryl in openstack-meeting17:07
Sam_swift_RackspHey guys. Daryl had a product emergency and could not attend and Jay is out this week17:07
*** anderstj_ has joined #openstack-meeting17:08
davidkranzHmm. I got an email from Jay about 1/2 hour ago.17:08
Sam_swift_RackspI have the overall agenda that Daryl was planning if we want to move forward17:08
davidkranzLet's see what we can cover.17:08
Sam_swift_Rackspcool17:08
Sam_swift_RackspFirst topic from Daryl's agenda was Status of Swift tests17:09
Ravikumar_hpi am interested in it17:09
*** Carlos_Swift has joined #openstack-meeting17:09
JoseSwiftQEAh yes.  I submitted my changes on the 1st.  They're up for review now17:09
JoseSwiftQEhttps://review.openstack.org/#/c/7465/17:09
Ravikumar_hpJoseSwiftQE: thanks17:09
davidkranzJoseSwiftQE: Do the tests skip if there is no swift endpoint?17:10
*** winston-d has joined #openstack-meeting17:10
JoseSwiftQEThey're not decorated as such, and I didn't test to see if it happens automagically.17:10
Ravikumar_hpJoseSwiftQE: i think we can do all enhancements once basic tests are in place17:11
davidkranzJoseSwiftQE: You couldl do the same thing Jay did recently with the Quantum network tests17:11
*** anderstj has quit IRC17:11
*** dachary has quit IRC17:12
*** dachary has joined #openstack-meeting17:12
JoseSwiftQEhttps://github.com/openstack/tempest/blob/master/tempest/tests/network/base.py17:14
JoseSwiftQEthat right?17:14
*** bencherian has quit IRC17:14
davidkranzJoseSwiftQE: Yes.17:15
davidkranzSince Daryl has reviewed the swift stuff I guess he should give final approval.17:15
JoseSwiftQESure.17:16
*** jakedahn is now known as jakedahn_zz17:16
Sam_swift_RackspSeems like the plan. It makes sense to let Daryl have the final approval, get the tests in, then submit a patch to make the final upgrade. These tests have been waiting a long time to get checked in. :-)17:17
JoseSwiftQEI'd like to get all the base stuff in first...exactly that ^^ :D17:17
davidkranzSam_swift_Racksp: I agree, but am not sure what the hold up was.17:17
Sam_swift_RackspOK. So next topic was Status of Parallelization modifications but that was for Daryl so let's skip to outstanding code reviews17:17
Sam_swift_RackspI'm not sure either, I'll follow up with Daryl after the meeting17:18
Sam_swift_Racksptopic: Outstanding code reviews17:18
davidkranzJay said yesterday he was going to run the diablo fixes and approve them.17:18
Sam_swift_RackspGood deal.17:19
Sam_swift_RackspIt looks like there are 15 outstanding code reviews17:19
Sam_swift_Racksphttps://review.openstack.org/#/q/status:open+project:openstack/tempest,n,z17:19
davidkranzI think only a few of them are actually waiting for a new review.17:20
Sam_swift_Rackspcorrec17:20
Sam_swift_Rackspcorrect17:20
Sam_swift_Racksp6 are waiting a new review17:20
davidkranzMy draft is more waiting for a discussion which I guess is not going to happen today.17:21
davidkranzUnless any of you have any opinion to express about that issue.17:21
Sam_swift_RackspI have some thoughts on it as well for sure but not sure if we should wait for Daryl and/or Jay to be in the meeting with us. :-)17:21
Sam_swift_Rackspthough I17:22
Sam_swift_Rackspthough I'm happy to discuss if you want17:22
Sam_swift_RackspI agree that resource management should be taken care of and automated so as not to be up to the test writer to explicitly deal with it. I just feel that it should be in a BaseTestClass as has been proposed/used so far17:23
davidkranzOK, I hear that people don't want resource collection on the rest client but that is the only place it can go right now that doesn't force the user to write extra boilerplate code as far as I can see.17:23
davidkranzWhat is the alternative?17:23
Sam_swift_RackspI haven't reviewed the code fully yet, however, in a general concept I have used in the past, I have used a manager of sorts (which I think Jay proposed something similar not long ago) to abstract between the test case itself and the actual client17:26
*** rash has joined #openstack-meeting17:26
davidkranzI made a comment in https://review.openstack.org/#/c/8193/ that we could introduce a ResourceTracker class that the various clients could inherit.17:26
davidkranzThat would be kind of like a manager as you describe.17:27
Sam_swift_RackspLet me take a look, just a sec17:27
davidkranzThe second to last comment.17:28
Sam_swift_Rackspthat makes some sense to me (the second to the last comment) I guess my real issue is with the client owning the cleanup logic, I.E. what if the test needs to implicitly leave a resource around (though I guess that can be a slippery slope as well)17:30
Sam_swift_Rackspsome kind of tracker/observer that was there but the ultimate decision of what to delete/free remained in the base test case or test runner class17:31
davidkranzSam_swift_Racksp: That is what the proposal does. It is up to the test class to call the cleanup and it can manually free resources as well.17:31
Sam_swift_Rackspthe way I've usually built a lot of frameworks is kinda TestRunner/Controller --> TestCase --> Facade --> Client --> Connector --> AUT17:31
*** Mandell has joined #openstack-meeting17:32
JoseSwiftQEThere's precedent for that actually.  Swift Access log delivery requires things to stick around for a bit.17:32
Sam_swift_Rackspdavidkranz: aha! I'll keep this and read it more closely after the meeting17:32
Sam_swift_Rackspit could be we are talking about the same thing but I'm playing some catch up. :-)17:32
JoseSwiftQESo, if things where deleted automatically, access log would never be delivered.17:32
Sam_swift_Rackspin that TestRunner/Controller --> TestCase --> Facade --> Client --> Connector --> AUT model the Test Case owns the ultimate decision on pass/fail, clean-up, etc... but the default behavior of a test case is to always clean up and fail on any unexpected error/exception17:33
davidkranzSam_swift_Racksp: Sure. Right now the free is in the tearDown and the test would have to override but I would be happy to change that.17:33
davidkranzSam_swift_Racksp: I meant the BaseTest tearDown.17:34
davidkranzSam_swift_Racksp: In that case I think it is already doing exactly what you are suggesting.17:34
davidkranzLet's see what Jay and Daryl have to say about my comment. I think I was not clear enough at the beginning about why the RestClient needed to be involved at all.17:35
Sam_swift_Racksphmmm.. I think you are right in talking with you. Let me run this one by Daryl and yes I'm thinking you and I are likely on the same page David17:36
Sam_swift_RackspI'll follow up on this one as well with Daryl.17:37
davidkranzJoseSwiftQE: It may be that swift tests would specify a different behavior.17:37
davidkranzJoseSwiftQE: Or turn off resource tracking if that makes sense.17:37
JoseSwiftQEYeah, that sounds good.  I can live with it so long as it's overridable for edgecases like mine.17:37
Sam_swift_Rackspwhich the swift test could do if it over-rides the cleanup method in BaseTestCase17:37
*** dendrobates is now known as dendro-afk17:38
Sam_swift_RackspMight be a good thing to have the resource tracking enabled by default but turned off with a config or passed bool as well. That way we give ultimate control to the test case but without implicit action the resources are automatically tracked and scrubbed.17:38
Sam_swift_Racksp:-)17:38
Sam_swift_RackspI17:38
Sam_swift_RackspI'll stop fat fingering my enter button. :-D17:38
davidkranzSounds good.17:38
Sam_swift_RackspI'll make some comments on your page David.17:39
davidkranzSam_swift_Racksp: Thanks.17:39
Sam_swift_RackspNo worries. I used Eclipse/PyDev by the way too. :-)17:39
Sam_swift_RackspOk. So next topic17:40
davidkranzDoes any one have comments about the issue of how many argument variants a Tempest test should have?17:40
Sam_swift_RackspI'm sorry, not sure about that one. Is there a link I could play catch up on?17:40
*** jakedahn_zz is now known as jakedahn17:41
*** dwcramer has joined #openstack-meeting17:41
davidkranzJust my comments in this bug ticket https://bugs.launchpad.net/tempest/+bug/101001417:41
uvirtbotLaunchpad bug 1010014 in tempest "Test Cases: Negative test cases for Server Metadata" [Undecided,In progress]17:41
*** littleidea has quit IRC17:43
*** edygarcia has quit IRC17:45
*** gyee has joined #openstack-meeting17:45
Sam_swift_Racksphmmm...That one is a big can of worms in one sense, however, I agree that we should have the break between what a unit test does and what a test case does17:46
davidkranzI fully understand the desire to have a set of tests that can be pointed at a real deployment and provide "acceptence" but we need to decide how much we are going to trust unit tests as a surrogate for that.17:46
Sam_swift_Racksptotally agree with you. :-)17:46
Sam_swift_Rackspnot sure what the plan is for that though becuase I have shot myself in the foot in the past at other companies when I got to trust the unit tests too much. :-)17:46
Ravikumar_hpdavidkranz: agree. we will linit those boundaty tests17:47
davidkranzSam_swift_Racksp: Yes. In an ideal world the specification of test cases for an API would be more abstract and both unit and functional tests could be generated.17:47
davidkranzI think that is a bit much to chew off now.17:48
*** winston-d has quit IRC17:49
*** dendro-afk is now known as dendrobates17:50
davidkranzIt is really a matter of architectural knowlege and judgement that really the architects of the AUT know best.17:50
Sam_swift_Rackspagree. This might be a much larger discussion, though we do need to make a constant vigil (at the risk of sounding dramatic) that the tests we add are adding value at each inclusion17:50
*** edygarcia has joined #openstack-meeting17:50
davidkranzFor now I just think our time would be better spent on stressing the system and doing things that really are useless for unit tests.17:51
davidkranzLike creating a stress test for the API rather than adding more argument combinations.17:51
Sam_swift_RackspAgreed. We really should be looking at user scenario's and more real world stress situations than nit-picking every parameter path. :-)17:52
davidkranzSam_swift_Racksp: It is pretty easy to add new cases to the tempest/stress tests and I would be happy to help with any issues if some one tried to do this.17:52
davidkranzDo we want to discuss the last issue which was tempest execution time?17:54
davidkranzIf not, I guess we can close the meeting.17:58
Ravikumar_hpdavidkranz: tempest tests are growing17:58
JoseSwiftQEWe probably won't be able to speak to that w/o Daryl at least.17:58
davidkranzRavikumar_hp: I know. That is why I have been flaming about issues surrounding that!17:59
*** novas0x2a|laptop has quit IRC17:59
Sam_swift_RackspI agree with Jose. We should wait for Daryl on that one.17:59
davidkranzMe too.17:59
Ravikumar_hpok.17:59
Sam_swift_RackspHe does have a few ideas and I know he has some strong feelings in general17:59
*** edygarcia has quit IRC18:00
davidkranzSam_swift_Racksp: Perhaps he can send an email summarizing his feelings and perhaps a proposal to move forward.18:00
Sam_swift_RackspCool. I'll take that as an action item to speak with Daryl.18:01
Sam_swift_Rackspso as a recap:18:01
Sam_swift_RackspI'm going to follow up with Daryl to get the swift tests approved/integrated18:01
Sam_swift_Rackspafter swift tests are aprovved, Jose is going to make some minor updates ot the base tests for swift18:01
*** bswartz has joined #openstack-meeting18:02
Sam_swift_RackspI'm going to follow up with some comments on the draft of resource management from David summarizing our discussion here18:02
Sam_swift_Rackspand I'm going to follow up with Daryl requesting an email to the group on tempest execution time18:03
davidkranzThanks, Sam.18:03
*** donaldngo_hp has quit IRC18:04
davidkranzUnfortunately no one remembered to start the meeting so there won't be a stand-alone log.18:04
Sam_swift_Rackspno worries David. After talking I think you and I are on the same page (or at least close enough) on the whole resource management issue and I believe Daryl is also really thinking the same thing, we were just talking about it in three different ways. :-)18:04
Sam_swift_Rackspack! Yes. I need to get a list of the IRC commands needed. :-)18:04
davidkranzSam_swift_Racksp: I forgot about http://wiki.debian.org/MeetBot18:04
Sam_swift_RackspI'll send a transcript to Daryl in an email and he can get it in the right place hopefully. :-)18:05
davidkranzSam_swift_Racksp: Great. See y'all next week.18:05
Sam_swift_RackspAh. Perfect. Thanks David. I'll be better prepared next time. :-) See everyone next week!18:05
*** Sam_swift_Racksp has quit IRC18:07
*** markmcclain has quit IRC18:13
*** markmcclain has joined #openstack-meeting18:14
*** Ravikumar_hp has quit IRC18:17
*** rash has quit IRC18:19
*** donaldngo_hp has joined #openstack-meeting18:20
*** Carlos_Swift has quit IRC18:24
*** johnpostlethwait has joined #openstack-meeting18:26
*** littleidea has joined #openstack-meeting18:27
*** novas0x2a|laptop has joined #openstack-meeting18:33
*** dhellmann has quit IRC18:37
*** JoseSwiftQE has left #openstack-meeting18:39
*** edygarcia has joined #openstack-meeting18:49
*** gyee has quit IRC18:51
*** arunkant has joined #openstack-meeting18:54
*** edygarcia has quit IRC18:58
*** s0mik has quit IRC19:00
*** s0mik has joined #openstack-meeting19:10
*** somik has joined #openstack-meeting19:11
*** s0mik has quit IRC19:11
*** somik is now known as s0mik19:11
*** bencherian has joined #openstack-meeting19:11
*** dabo has quit IRC19:15
*** dabo has joined #openstack-meeting19:16
*** jog0 has joined #openstack-meeting19:16
*** rash has joined #openstack-meeting19:30
*** lloydde has joined #openstack-meeting19:40
*** s0mik has quit IRC19:48
*** rash has quit IRC19:49
*** dabo has left #openstack-meeting19:52
*** ncode has joined #openstack-meeting19:56
*** gyee has joined #openstack-meeting20:00
*** lloydde has quit IRC20:06
*** lloydde has joined #openstack-meeting20:08
*** Guest52738 has quit IRC20:09
*** nikhil_ has joined #openstack-meeting20:09
*** dwcramer has quit IRC20:09
*** nikhil_ is now known as Guest9530620:09
*** bencherian has quit IRC20:11
*** dolphm_ has quit IRC20:13
*** dabo has joined #openstack-meeting20:16
*** dabo has quit IRC20:21
*** dwcramer has joined #openstack-meeting20:21
*** bswartz has left #openstack-meeting20:22
*** s0mik has joined #openstack-meeting20:22
*** ttrifonov is now known as ttrifonov_zZzz20:33
*** ttrifonov_zZzz is now known as ttrifonov20:34
*** s0mik has quit IRC20:35
*** lloydde has quit IRC20:37
*** s0mik has joined #openstack-meeting20:38
*** Guest95306 has quit IRC20:39
*** hggdh has quit IRC20:40
*** maoy has joined #openstack-meeting20:40
*** s0mik has quit IRC20:40
*** dabo has joined #openstack-meeting20:41
*** dwcramer has quit IRC20:41
*** hggdh has joined #openstack-meeting20:43
*** joesavak has quit IRC20:47
*** tongli has quit IRC20:49
*** s0mik has joined #openstack-meeting20:50
*** s0mik has quit IRC20:53
*** gyee has quit IRC20:54
*** primeministerp is now known as primeminsterp|af20:56
*** s0mik has joined #openstack-meeting20:57
*** primeminsterp|af is now known as primeministerp20:57
*** s0mik has quit IRC21:02
*** s0mik has joined #openstack-meeting21:11
*** dwcramer has joined #openstack-meeting21:13
*** joearnold has quit IRC21:13
*** dendrobates is now known as dendro-afk21:21
*** dwcramer has quit IRC21:22
*** s0mik has quit IRC21:23
*** dmd17 has joined #openstack-meeting21:26
*** s0mik has joined #openstack-meeting21:27
*** maoy has quit IRC21:28
*** dachary has quit IRC21:29
*** dachary has joined #openstack-meeting21:29
*** s0mik has quit IRC21:30
*** s0mik has joined #openstack-meeting21:34
*** markvoelker has quit IRC21:41
*** joearnold has joined #openstack-meeting21:53
*** semyazz has joined #openstack-meeting21:55
*** semyazz has left #openstack-meeting21:55
*** lloydde has joined #openstack-meeting21:56
*** ywu has joined #openstack-meeting21:58
*** lloydde has quit IRC22:00
*** ayoung has quit IRC22:00
*** ryanpetrello has quit IRC22:08
*** blamar has quit IRC22:20
*** dmd17 has quit IRC22:20
*** bencherian has joined #openstack-meeting22:23
*** dwcramer has joined #openstack-meeting22:26
*** ncode has quit IRC22:28
*** markmcclain1 has joined #openstack-meeting22:32
*** markmcclain has quit IRC22:32
*** dachary has quit IRC22:33
*** danwent has quit IRC22:35
*** adjohn has joined #openstack-meeting22:35
*** s0mik has quit IRC22:36
*** dwcramer has quit IRC22:37
*** s0mik has joined #openstack-meeting22:37
*** mattray has quit IRC22:46
*** anderstj_ has quit IRC22:49
*** nikhil_ has joined #openstack-meeting22:53
*** nikhil_ is now known as Guest867922:53
*** ncode has joined #openstack-meeting23:12
*** sleepsonthefloor is now known as sleepsonzzz23:15
*** ttrifonov is now known as ttrifonov_zZzz23:15
*** Guest8679 has quit IRC23:18
*** ywu has quit IRC23:25
*** dwcramer has joined #openstack-meeting23:26
*** danwent has joined #openstack-meeting23:35
*** jakedahn is now known as jakedahn_zz23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!