Friday, 2013-08-02

*** ErikB has quit IRC00:01
*** markwash has quit IRC00:08
*** NikitaKonovalov has joined #openstack-meeting-alt00:13
*** IlyaE has joined #openstack-meeting-alt00:16
*** NikitaKonovalov has quit IRC00:17
*** tanisdl has quit IRC00:18
*** benl_ has joined #openstack-meeting-alt00:30
*** Riddhi has quit IRC00:33
*** benl_ has quit IRC00:34
*** flwang has quit IRC01:14
*** Riddhi has joined #openstack-meeting-alt01:16
*** markmcclain has quit IRC01:17
*** IlyaE has quit IRC01:21
*** sarob has joined #openstack-meeting-alt01:33
*** ben_duyujie has quit IRC01:35
*** sarob has quit IRC01:37
*** anteaya has quit IRC02:07
*** NikitaKonovalov has joined #openstack-meeting-alt02:14
*** NikitaKonovalov has quit IRC02:18
*** flwang has joined #openstack-meeting-alt02:19
*** Riddhi has quit IRC02:46
*** pcm_ has quit IRC02:48
*** esp has joined #openstack-meeting-alt02:51
*** esp has quit IRC03:02
*** flwang1 has joined #openstack-meeting-alt03:08
*** flwang has quit IRC03:09
*** NikitaKonovalov has joined #openstack-meeting-alt03:14
*** sarob has joined #openstack-meeting-alt03:18
*** NikitaKonovalov has quit IRC03:19
*** sarob_ has joined #openstack-meeting-alt03:47
*** sarob has quit IRC03:50
*** sarob_ has quit IRC04:04
*** SergeyLukjanov has joined #openstack-meeting-alt04:12
*** flwang1 has quit IRC04:13
*** NikitaKonovalov has joined #openstack-meeting-alt04:15
*** NikitaKonovalov has quit IRC04:19
*** basha has joined #openstack-meeting-alt04:23
*** flwang has joined #openstack-meeting-alt04:23
*** dina_belova has joined #openstack-meeting-alt04:26
*** akuznetsov has joined #openstack-meeting-alt04:39
*** IlyaE has joined #openstack-meeting-alt04:43
*** boris-42 has joined #openstack-meeting-alt04:47
*** dina_belova has quit IRC04:49
*** flwang1 has joined #openstack-meeting-alt05:01
*** flwang has quit IRC05:02
*** zane1 has joined #openstack-meeting-alt05:03
*** IlyaE has quit IRC05:07
*** IlyaE has joined #openstack-meeting-alt05:14
*** NikitaKonovalov has joined #openstack-meeting-alt05:15
*** NikitaKonovalov has quit IRC05:20
*** akuznetsov has quit IRC05:21
*** jmaron1 has joined #openstack-meeting-alt05:29
*** jmaron has quit IRC05:30
*** akuznetsov has joined #openstack-meeting-alt05:31
*** Nadya has joined #openstack-meeting-alt05:34
*** mestery has quit IRC05:34
*** IlyaE has quit IRC05:43
*** jmaron1 has quit IRC05:44
*** dina_belova has joined #openstack-meeting-alt05:46
*** dina_belova has quit IRC05:48
*** jmaron has joined #openstack-meeting-alt05:48
*** Nadya has quit IRC05:51
*** lastidiot has quit IRC05:55
*** IlyaE has joined #openstack-meeting-alt05:56
*** SergeyLukjanov has quit IRC06:08
*** akuznetsov has quit IRC06:16
*** basha has quit IRC06:19
*** salv-orlando has joined #openstack-meeting-alt06:20
*** basha has joined #openstack-meeting-alt06:24
*** NikitaKonovalov has joined #openstack-meeting-alt06:25
*** flwang1 has quit IRC06:31
*** basha has quit IRC06:46
*** basha has joined #openstack-meeting-alt06:47
*** boris-42 has quit IRC06:48
*** basha has quit IRC06:54
*** basha has joined #openstack-meeting-alt07:00
*** Nadya has joined #openstack-meeting-alt07:07
*** IlyaE has quit IRC07:14
*** SergeyLukjanov has joined #openstack-meeting-alt07:18
*** NikitaKonovalov has quit IRC07:22
*** basha has quit IRC07:27
*** Nadya has quit IRC07:30
*** basha has joined #openstack-meeting-alt07:34
*** mestery has joined #openstack-meeting-alt07:47
*** NikitaKonovalov has joined #openstack-meeting-alt07:50
*** mestery has quit IRC07:58
*** dina_belova has joined #openstack-meeting-alt07:59
*** dina_belova has quit IRC08:00
*** boris-42 has joined #openstack-meeting-alt08:02
*** dina_belova has joined #openstack-meeting-alt08:13
*** akuznetsov has joined #openstack-meeting-alt08:13
*** SergeyLukjanov has quit IRC08:28
*** NikitaKonovalov has quit IRC08:33
*** zane1 has quit IRC08:45
*** dmitryme has joined #openstack-meeting-alt08:51
*** basha has quit IRC09:01
*** dina_belova has quit IRC09:07
*** dmitryme has left #openstack-meeting-alt09:17
*** basha has joined #openstack-meeting-alt09:17
*** basha_ has joined #openstack-meeting-alt09:20
*** basha has quit IRC09:22
*** basha_ is now known as basha09:22
*** akuznetsov has quit IRC09:32
*** NikitaKonovalov has joined #openstack-meeting-alt09:33
*** NikitaKonovalov has quit IRC09:37
*** nmakhotkin has quit IRC09:42
*** dina_belova has joined #openstack-meeting-alt09:47
*** NikitaKonovalov has joined #openstack-meeting-alt09:48
*** mestery has joined #openstack-meeting-alt09:52
*** benl has joined #openstack-meeting-alt09:54
*** benl_ has joined #openstack-meeting-alt09:55
*** benl has quit IRC09:55
*** dina_belova has quit IRC10:00
*** basha has quit IRC10:06
*** pcm_ has joined #openstack-meeting-alt10:09
*** pcm_ has joined #openstack-meeting-alt10:09
*** basha has joined #openstack-meeting-alt10:12
*** akuznetsov has joined #openstack-meeting-alt10:25
*** Riddhi has joined #openstack-meeting-alt10:25
*** zane has joined #openstack-meeting-alt10:33
*** basha has quit IRC10:38
*** dina_belova has joined #openstack-meeting-alt10:39
*** bourke has joined #openstack-meeting-alt10:42
*** Riddhi has quit IRC10:47
*** zane1 has joined #openstack-meeting-alt10:47
*** Riddhi has joined #openstack-meeting-alt10:49
*** zane has quit IRC10:49
*** benl_ has quit IRC10:52
*** demorris has joined #openstack-meeting-alt10:58
*** Riddhi_ has joined #openstack-meeting-alt11:18
*** Riddhi has quit IRC11:20
*** Riddhi_ is now known as Riddhi11:20
*** benl has joined #openstack-meeting-alt11:23
*** akuznetsov has quit IRC11:27
*** NikitaKo_ has joined #openstack-meeting-alt11:34
*** NikitaKonovalov has quit IRC11:37
*** Riddhi has quit IRC11:46
*** NikitaKo_ has quit IRC11:49
*** NikitaKonovalov has joined #openstack-meeting-alt11:50
*** ruhe has joined #openstack-meeting-alt11:59
*** demorris has quit IRC11:59
*** dina_belova has quit IRC12:08
*** akuznetsov has joined #openstack-meeting-alt12:21
*** anteaya has joined #openstack-meeting-alt12:23
*** ErikB has joined #openstack-meeting-alt12:28
*** demorris has joined #openstack-meeting-alt12:31
*** dina_belova has joined #openstack-meeting-alt12:34
*** SergeyLukjanov has joined #openstack-meeting-alt12:35
*** vkmc has joined #openstack-meeting-alt12:36
*** vkmc has joined #openstack-meeting-alt12:36
*** pdmars has joined #openstack-meeting-alt12:40
*** akuznetsov has quit IRC12:43
*** kevinconway has quit IRC12:44
*** kevinconway has joined #openstack-meeting-alt12:47
*** ErikB has quit IRC12:50
*** kevinconway has quit IRC12:54
*** SergeyLukjanov has quit IRC12:54
*** ruhe has quit IRC12:59
*** ruhe has joined #openstack-meeting-alt13:00
*** kevinconway has joined #openstack-meeting-alt13:00
*** flwang has joined #openstack-meeting-alt13:00
*** dina_belova has quit IRC13:08
*** kevinconway has quit IRC13:09
*** zane1 has quit IRC13:10
*** kevinconway has joined #openstack-meeting-alt13:10
*** Riddhi has joined #openstack-meeting-alt13:21
*** akuznetsov has joined #openstack-meeting-alt13:21
*** aignatov has quit IRC13:21
*** aignatov has joined #openstack-meeting-alt13:22
*** markmcclain has joined #openstack-meeting-alt13:23
*** ErikB has joined #openstack-meeting-alt13:24
*** kevinconway has quit IRC13:26
*** djohnstone has joined #openstack-meeting-alt13:27
*** SergeyLukjanov has joined #openstack-meeting-alt13:28
*** dina_belova has joined #openstack-meeting-alt13:28
*** kevinconway has joined #openstack-meeting-alt13:32
*** ruhe has quit IRC13:36
*** benl has quit IRC13:36
*** pdmars_ has joined #openstack-meeting-alt13:36
*** pdmars has quit IRC13:36
*** NikitaKonovalov has quit IRC13:46
*** akuznetsov has quit IRC13:50
*** jmontemayor has joined #openstack-meeting-alt13:51
*** ErikB has quit IRC13:53
*** ErikB has joined #openstack-meeting-alt13:54
*** HenryG has joined #openstack-meeting-alt13:56
*** ErikB has quit IRC13:57
*** Riddhi_ has joined #openstack-meeting-alt13:57
*** ErikB has joined #openstack-meeting-alt13:57
*** markwash has joined #openstack-meeting-alt13:58
*** ruhe has joined #openstack-meeting-alt13:58
*** markwash_ has joined #openstack-meeting-alt13:59
*** Riddhi has quit IRC14:00
*** Riddhi_ is now known as Riddhi14:00
*** mclaren has joined #openstack-meeting-alt14:00
*** markwash has quit IRC14:02
*** markwash_ is now known as markwash14:02
*** dina_belova has quit IRC14:02
*** mclaren has quit IRC14:03
*** dina_belova has joined #openstack-meeting-alt14:03
*** akuznetsov has joined #openstack-meeting-alt14:09
*** basha has joined #openstack-meeting-alt14:12
*** sergmelikyan has quit IRC14:15
*** NikitaKonovalov has joined #openstack-meeting-alt14:16
*** lastidiot has joined #openstack-meeting-alt14:18
*** sergmelikyan has joined #openstack-meeting-alt14:23
*** basha has quit IRC14:28
*** megan_w has joined #openstack-meeting-alt14:33
*** megan_w has joined #openstack-meeting-alt14:34
*** lastidiot has quit IRC14:34
*** anteaya is now known as anteayah14:36
*** benl has joined #openstack-meeting-alt14:37
*** benl has quit IRC14:44
*** markmcclain has quit IRC14:44
*** SergeyLukjanov has quit IRC14:47
*** stanlagun has quit IRC14:51
*** iccha is now known as itcha-friday-fun14:51
*** SergeyLukjanov has joined #openstack-meeting-alt14:56
*** IlyaE has joined #openstack-meeting-alt14:58
*** dina_belova has quit IRC14:58
*** dina_belova has joined #openstack-meeting-alt14:59
*** stanlagun has joined #openstack-meeting-alt14:59
*** dina_belova has quit IRC15:02
*** NikitaKo_ has joined #openstack-meeting-alt15:03
*** pdmars_ has quit IRC15:03
*** pdmars has joined #openstack-meeting-alt15:03
*** NikitaKo_ has quit IRC15:04
*** NikitaKonovalov has quit IRC15:04
*** NikitaKonovalov has joined #openstack-meeting-alt15:04
*** dina_belova has joined #openstack-meeting-alt15:04
*** lastidiot has joined #openstack-meeting-alt15:11
*** mordred is now known as TheRonin15:11
*** malini has joined #openstack-meeting-alt15:12
*** NikitaKonovalov has quit IRC15:12
*** markwash has quit IRC15:13
*** markmcclain has joined #openstack-meeting-alt15:15
*** boris-42 has quit IRC15:21
*** tedross has joined #openstack-meeting-alt15:27
*** NikitaKonovalov has joined #openstack-meeting-alt15:30
*** zane1 has joined #openstack-meeting-alt15:31
*** zane1 has quit IRC15:36
*** markwash has joined #openstack-meeting-alt15:47
*** akuznetsov has quit IRC15:54
*** NikitaKonovalov has quit IRC15:55
*** itcha-friday-fun is now known as iccha15:58
*** whenry has joined #openstack-meeting-alt15:59
*** flaper87 has joined #openstack-meeting-alt15:59
*** Sriram has joined #openstack-meeting-alt15:59
*** cppcabrera has joined #openstack-meeting-alt16:02
flaper87#startmeeting marconi16:02
openstackMeeting started Fri Aug  2 16:02:21 2013 UTC and is due to finish in 60 minutes.  The chair is flaper87. Information about MeetBot at http://wiki.debian.org/MeetBot.16:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:02
*** openstack changes topic to " (Meeting topic: marconi)"16:02
openstackThe meeting name has been set to 'marconi'16:02
cppcabreraAwesome.16:02
Sriram:)16:02
flaper87Yo yo guys!!!!!!16:02
flaper87#topic agenda16:02
Sriramyo! :P16:02
*** openstack changes topic to "agenda (Meeting topic: marconi)"16:02
flaper87#link https://wiki.openstack.org/wiki/Meetings/Marconi#Agenda16:02
flaper87so, that's the agenda for today's meeting16:03
*** megan_w has quit IRC16:03
flaper87lets get it started16:03
*** zyuan has joined #openstack-meeting-alt16:03
cppcabreraTime to review some action items. :D16:03
maliniCan we also add a new meeting time to the agenda, plz?16:03
flaper87#topic review action items16:03
*** openstack changes topic to "review action items (Meeting topic: marconi)"16:03
flaper87#link http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-07-18-19.05.html16:03
flaper87cppcabrera: to turn apiclient-marconi etherpad into bps/bugs16:03
*** benl has joined #openstack-meeting-alt16:03
flaper87I should've been more involved in that, sorry16:04
cppcabreraI don't believe I got around to that.16:04
*** akuznetsov has joined #openstack-meeting-alt16:04
flaper87any news about that ?16:04
flaper87ok, lets schedule it for the next meeting16:04
cppcabreraI may have done some Trello'ing, but that's a blur to me at the moment.16:04
flaper87#action cppcabrera flaper87 to turn apiclient-marconi etherpad into bps/bugs16:04
cppcabrera+116:04
flaper87cppcabrera: to tie up loose ends re python-marconicient v1 design16:04
flaper87?16:04
cppcabreraYes, there's been progress there. :)16:05
cppcabrera#link https://wiki.openstack.org/wiki/Python_Marconi_Client16:05
*** marcos-sb has joined #openstack-meeting-alt16:05
cppcabreraI've since added some ideas on pubsub design.16:05
cppcabreraI still have some concerns.16:05
flaper87cppcabrera: awesome, thanks!16:05
cppcabreraI want to talk about openstack sdk/cli design with someone who's written such clients.16:05
cppcabreraFor example, I'm a little dubious about the naming of methods.16:06
cppcabreralist16:06
cppcabreraget16:06
cppcabrerapost16:06
flaper87Is this something that worths a thread in the m-l ?16:06
cppcabreraIt's... very HTTP-targeted, when I, as a user of the SDK, think it terms of resources.16:06
cppcabreraLike client.queues, etc.16:06
cppcabreraI believe so.16:06
cppcabreras/it/in16:06
flaper87We can get some of the best minds around this design process16:06
tedrossm-l?16:07
cppcabreraI'll start that discussion on the mailing list.16:07
flaper87tedross: mailing list16:07
flaper87awesome! thanks!16:07
cppcabreraopenstack-dev, woot. :)16:07
flaper87flaper87: to investigate mongo insert performance16:07
flaper87so, I did, I've some work done on that area16:07
cppcabrera#action cppcabrera to ask about client design on mailing list16:07
whenrycppcabrera, pls feel free to reachout to tedross or me re client apis16:07
flaper87there's still some work left there16:07
cppcabrerawhenry: Thanks!16:08
tedrosswe'll look at the link16:08
whenrylet us know how we can help16:08
Sriramwoot! awesome.16:08
flaper87tedross: whenry awesome, thanks guys!16:08
cppcabreraYes, much appreciated. :)16:08
flaper87last action item was: to finalize placement service strategy16:08
*** akuznetsov has quit IRC16:09
* cppcabrera looks for the wiki page...16:09
flaper87I guess cppcabrera worked more on that one and we'll discuss it in the second part of this meeting16:09
flaper87lets get to the next topic and then go back there16:09
SriramThere was quite a discussion on that. Lots of progress there :)16:09
whenrypls let us know the process: e.g. is it, opesntac-dev [marconi] for banter around ideas and then someone will update the docs?16:09
cppcabrera#link https://wiki.openstack.org/wiki/Marconi/bp/placement-service16:09
whenryoops sorry off topic16:09
flaper87whenry: we discuss most of the things on #openstack-marconi16:10
flaper87but, if there are some discussions that worth bringing up to the m-l, then we do it there16:10
cppcabreraI actually added our IRC channel to the official list a few minutes ago, heh. :P16:10
flaper87#topic Test re-factor and improvements16:10
*** openstack changes topic to "Test re-factor and improvements (Meeting topic: marconi)"16:10
whenryflaper87, ah, good to know.16:10
* whenry adds to favs16:10
flaper87cppcabrera: malini ^16:10
malinihttps://etherpad.openstack.org/marconi-test-refactoring16:10
flaper87Floor is all yours16:10
cppcabrera#link https://blueprints.launchpad.net/marconi/+spec/refactor-system-tests16:11
cppcabreraAwesome, malini.16:11
*** IlyaE has quit IRC16:11
* whenry closes #opensack-marconi and fixes typo 16:11
flaper87whenry: lol16:11
maliniCan we first take a stab at the long term vision, how we want the tests organized?16:11
flaper87malini: sure16:11
cppcabreraCurrently, I have some concerns about our functional tests. I think that as we're growing them now, they're getting a little fragile and need some DRYing up.16:12
maliniwe have had quite some discussions around this & have it documented in the etherpad16:12
malinicppcabrera: can you elaborate on the fragile part?16:12
malinimore specifics16:12
cppcabreramalini and I spoke about it a lot offline, and I'll elaborate on that for sure.16:12
flaper87malini: cppcabrera lets keep going on the structure and then get back there16:13
cppcabreraThere are two problems that I see wrt fragility.16:13
malini+116:13
maliniflaper87: +116:13
maliniw.r.t structure, lets get all the tests out of the core marconi code16:13
flaper87malini: AFAIS, the structure in the etherpad is much like we discussed, right?16:13
cppcabrera1. There are hidden ordering dependencies in the test at the moment. They've been crafted up to this point very carefully, and I see that. However, when others start contributing  tests, they might not be aware of the conventions and introduce subtle breaks.16:14
flaper87am I missing something ?16:14
*** benl_ has joined #openstack-meeting-alt16:14
*** IlyaE has joined #openstack-meeting-alt16:14
flaper87malini: ^16:14
maliniflaper87: yes, + we did some pokng around other OS repos & came up with what we have in the etherpad16:14
cppcabrera2. The tests allocate resources and don't guarantee clean up of those resources in the presence of test assertion failures.16:15
cppcabreraThat's it for fragility.16:15
zyuanthose are bad for testing16:15
zyuanthat's why i +1 for using context manager16:15
malinicppcabrera, zyuan: lets first address the structure part first16:15
flaper87malini: I think the structure in the etherpad makes sense16:15
*** ruhe has quit IRC16:15
malinidoes everybody agree on the etherpad?16:15
cppcabreraAlright, structure. :)16:15
clarkbzyuan: the fixtures lib gives you context manager for free16:15
clarkbzyuan: with a bunch of common fixtures baked in16:16
SriramYes, the structure looks good.16:16
zyuanclarkb: iknow. just... they are not common16:16
cppcabreraI think system should be on the same level as unit16:16
malinicppcabrera: as in not under v1??16:16
cppcabreraSo 'tests/{system, unit, smoke, ...}16:16
zyuanwhere is the etherpad?16:17
flaper87mmh, actually, I agree with cppcabrera16:17
cppcabrerazyuan: https://etherpad.openstack.org/marconi-test-refactoring16:17
*** benl has quit IRC16:17
flaper87then we should have {v1, v2} where needed16:17
flaper87i.e: transport16:17
malinicppcabrera: the argument somebody had for tht was unit tests won't change between versions16:17
cppcabreramalini: +1. The unit tests should evolve with the code.16:18
cppcabreraThey are version agnostic.16:18
flaper87cppcabrera: mmh, not sure about that16:18
*** akshayms has quit IRC16:18
zyuandoes system test use utils under tests/16:18
zyuan?16:18
flaper87If we make changes in the API, the API unittests for v1 will remain the same and then we'll have some for v216:18
cppcabrerazyuan: Not yet.16:18
flaper87so, we need v1 and v2 under transport16:19
cppcabreraflaper87: I see what you're saying. It makes sense to me. I typically envision unit test structure mirroring implementation structure.16:19
cppcabreraI'm less certain about the structure of functional/system tests.16:19
malinithe functional tests are independent of the code structure16:20
cppcabrera+1 malini16:20
maliniapplication code structure*16:20
cppcabreraI feel like they should mirror the endpoints, instead.16:20
flaper87so, what about unit and functional in the same level and then adding {v1,v2} under functional instead ?16:20
maliniit should be more in terms of functionality16:20
flaper87to keep consistency in the structure16:20
flaper87type/functionality/version16:21
flaper87unit/transport/v116:21
maliniflaper87: +116:21
cppcabreraI like the idea of that.16:21
flaper87functional/transport/v116:21
cppcabrerafunctional/http/v1, functional/zmq/v1?16:21
flaper87we can either do that or just have test_impl_zmq.py16:22
flaper87and test_impl_wsgi.py16:22
flaper87I'm good with both16:22
*** ametts has joined #openstack-meeting-alt16:22
cppcabreraAlright, I'm +1 with the proposed structuring.16:22
Sriramyes.. functional/transport/http/v1 also seems good.16:22
zyuanfunctional/v1/test_impl_wsgi.py16:22
malinihmmm...16:22
zyuanfunctional tests should be at a very high level16:22
*** amitgandhi has joined #openstack-meeting-alt16:23
maliniyes..tht's why  we will not have functional/v1/test_impl_wsgi.py16:23
zyuanit does not know about "transport", IMO16:23
zyuanit only know "the procedures to use"16:23
maliniwe'll have functional/v1/test_queue.py,  functional/v1/test_messages.py,  functional/v1/test_claim.py etc.16:23
cppcabrera+1 malini. That matches up more clearly for me with the idea of testing endpoints.16:24
malinior  functional/v1/http/*  ??16:24
malini functional/v1/zmq/*  16:24
cppcabreraYeah, we might break it down by transport at some point.16:24
zyuan^^ yea16:24
malinietc. ??16:24
cppcabreraWhen we get there. :)16:24
zyuanif test_impl_wsgi is too large16:24
cppcabreras/might/should16:24
zyuanbreak it into http/16:24
flaper87+116:24
whenrywhy have version down inside the topology ?16:24
whenryinstead of higher up?16:24
whenrysorry I missed that16:25
flaper87whenry: consistency with the structure16:25
flaper87type/../../16:25
flaper87unit/../16:25
flaper87functional/...16:25
flaper87and so on16:25
whenryopenstack test structure ?16:25
flaper87not exactly16:25
flaper87most projects do that, though16:25
*** kgriffs has joined #openstack-meeting-alt16:25
malini whenry: were you referring to v1 ?16:25
whenryso type might have a diff version than functional?16:25
whenrymalini, yes16:25
flaper87plus, functional tests can be used for other things than just transport tests16:26
whenryor does v1 mean the same v across these ?16:26
flaper87what if we've a cache library we want to test ?16:26
maliniv1 is the application code version tht it'll test16:26
flaper87we might need some functional tests for that16:26
zyuanapi version16:26
flaper87that have to live under functional/16:26
flaper8716:26 UTC, Can we say we agreed on the structure ?16:27
cppcabrera+116:27
cppcabreraI'd like to discuss the test strategy re: fragility. ^16:27
flaper87#action malini to update the etherpad and start working on the re-factor16:27
flaper87cppcabrera: go ahead16:27
cppcabreraI mentioned previously that the tests are fragile atm. 1) hidden ordering dependencies and 2) resources allocated not guaranteed to be cleaned up in the presence of test assertion failures.16:28
* kgriffs is here - had to get a new chair and make it through the welcome gauntlet16:28
cppcabreraI have a patch pending that addresses these issues, and I'd like to gather thoughts on that.16:28
malinican I first address the two issues, before we discuss the patch ?16:29
cppcabreraCertainly, malini. :)16:29
maliniThe functional tests are designed to simulate user behavior16:30
malinii.e follow sequence of events like create queue, post message, claim them, delete messages etc. , sort of what we expect in a real world16:30
maliniso for message tests, the system tests have a pre-requisite to create a queue16:31
zyuanmalini: iirc, there is a testing tool to use a | separated table to discribe it?16:31
malinizyuan: you are refrring to robot framework ?16:31
zyuanyea16:32
maliniwe moved away from robot to nose, to make it easier for everybody to understand16:32
cppcabrera+1 to that, malini16:32
zyuanwe all understand the need of action dependency16:32
Sriram+1 for nose! :)16:32
flaper87+1 for nose16:32
maliniok..so back to what I was referring16:32
zyuanjust, such a dependency should not across tests16:32
cppcabreraI agree that there is a place for tests that simulate random user behavior, and can get messy.16:33
malinitht where we disgree16:33
zyuanyou can use context manager, fixture, or even very long tests if you want16:33
cppcabreraThanks to those tests, we were able to find the truncated JSON reply.16:33
cppcabreraMy argument is that the functional tests, or the system tests, are not the place for that.16:33
*** NikitaKonovalov has joined #openstack-meeting-alt16:33
maliniyes, & we found it because we had data from the prev test16:33
cppcabrerawe need some form of test that is highly reproducible and consistent.16:33
zyuani have a suggestion16:34
cppcabreraTo guarantee that reproducibility, we need to eliminate dependencies across test case boundaries.16:34
maliniwe found the truncated JSON because thesystem tests cud repro it consistently16:34
kgriffsIMHO random behavior is kind of like fuzzing in securty - it can find particular types of errors, but is probably less helpful than other types of tests.16:34
zyuanput test in non-test member-functions16:34
flaper87kgriffs: +116:34
maliniwe are not talking random behavior16:34
zyuanand call them randomly, explicitly16:34
zyuannamely, controlly messy16:34
maliniwe are talking abt using the API realistically16:35
kgriffsmalini: correct me if I'm wrong, but that is what the functional tests do today, right?16:35
malinikgriffs: can you clarify the what in ur question?16:35
flaper87I'd like to see functional tests being triggered by tox16:35
flaper87with other tests16:35
maliniflaper87: +116:35
kgriffssorry, just confirming16:35
*** tanisdl has joined #openstack-meeting-alt16:36
cppcabreratox -e nightly, tox -e smoke, tox -e system16:36
cppcabrera:)16:36
flaper87and IMHO, the functional tests suit should run marconi itself16:36
*** dina_belova has quit IRC16:36
maliniflaper87: +116:36
malini* seems to agree with everything flaper87 has'16:36
kgriffsI think if we want to do random behavior (with 2-3 seeds for repeatability) that is a new bp16:36
amitgandhito me random behaviour testing (or using api realistically) falls under load testing (where diff scenarios are fed as inputs)16:36
kgriffsand we should do it later16:36
flaper87Can we say that's the next step for our functional tests?16:36
cppcabreraamitgandhi, malini: Makes me think of tsung.16:36
maliniclarification: we are NOT testing random behavior in system tests now16:36
cppcabreramalini is right.16:37
flaper871) re-structure 2) run marconi 3) run with tox ?16:37
cppcabreraThe tests atm are NOT random.16:37
cppcabreraThey just have ordering dependencies.16:37
kgriffsright, I am just saying that we should not do a lot of work now to support that16:37
cppcabreraThey must execute as test_001, test_002, ...16:37
cppcabreraOr an error occurs.16:37
kgriffs(to support arbitrary test ordering)16:37
cppcabreraThe work is done, kgriffs. :P16:38
kgriffswe already have tsung doing some of that, albeit not repeatable randomness16:38
cppcabreraThe work is done to support arbitrary ordering, I should say,16:38
cppcabrera#link https://review.openstack.org/#/c/39369/16:38
kgriffscppcabrera: does it break linear testing?16:38
maliniwill come to the patch soon16:38
malinican we finish the discussion first?16:38
cppcabreramalini: Yes. :)16:39
cppcabrerakgriffs: Linear testing?16:39
kgriffssorry, ordered testing, whatever you want to call it16:39
kgriffsfunctional testing, I guess16:39
cppcabreraIs the "it" the patch?16:39
maliniSo if we want to keep the tests independent, we should first have make sure that the pre-requisites are covered16:39
malini'it' = patch16:40
cppcabreraCool.16:40
cppcabreraThe patch does not break ordered testing.16:40
maliniit doesnt make sense to have an independent list queues test that can run in parallel with create queue, if all list queue does is return a list with one ASCII queue16:40
cppcabreraCould you elaborate on the pre-reqs, malini?16:40
*** IlyaE has quit IRC16:41
flaper87I don't think functional tests should be independent16:41
malinihence the current ordering in today's tests16:41
flaper87I'm missing something, I guess16:41
maliniflaper87: +116:41
*** SergeyLukjanov has quit IRC16:41
cppcabreraI feel like I'm missing something, too...16:41
kgriffsok16:41
kgriffslet me see If can summarize16:41
cppcabreraI might be thinking of a different testing strategy that checks whether the API behaves as expected.16:41
cppcabreraVersus what a user might do..16:42
kgriffsit seems like we have two different kinds of tests in mind16:42
cppcabreraYes!16:42
kgriffsand trying to do it with a single set of tests16:42
maliniwe need to guarantee the same behavior on each test run, which we do today16:42
flaper8716:42 UTC (there won't be time for placement)16:42
maliniyeap, we are from different schools of thought16:42
maliniI am thinking user, cppcabrera is thinking code16:43
kgriffsI think it would be better to make a random-ish/load test as a separate thing16:43
kgriffsand factor out the common bits16:43
kgriffsthat can be shared with functional tests16:43
cppcabrerakgriffs: +116:43
flaper87kgriffs: +116:43
maliniwe dont have random-ish tests today16:43
flaper87but that's a third test suit for me16:43
kgriffsmalini: understood16:43
flaper87unit, functional. random/load16:44
kgriffsload is tricky16:44
flaper87unit, functional, random :P16:44
flaper87:D16:44
cppcabrera:P16:44
kgriffsI guess you would simulate kosher client behavior at the same time as naughty behavior16:44
kgriffs(to kind of simulate the real world)16:44
malinithe point of disagreement is, do the functional tests  (within a test suite) need to be independent of each other ?16:44
zyuani want them to be independent16:45
cppcabreraYeah, I think it comes down to that.16:45
kgriffsI don't think so, since by definition, they are dependent?16:45
kgriffsor16:45
zyuanfunctional tests are test16:45
zyuannot simulation16:45
kgriffsis it a matter of just refactoring so each test does just-in-time setup16:45
flaper87zyuan: +!16:45
flaper87zyuan: +116:45
cppcabreraThe dependencies are established at the suite level, and the test cases perform operations.16:45
flaper87kgriffs: yes, that's what needs to be done!16:46
maliniI think test suite shud be independent, no the tests themeselves16:46
flaper87malini: +116:46
maliniit makes sense for unit tests to be independent, not the functional tests16:46
cppcabreraUsing contextmanagers, JIT dep setup is easy. :D16:46
*** akuznetsov has joined #openstack-meeting-alt16:46
malinicppcabrera: what are we gaining by doing that ?16:46
amettsSeems like it would slow things down.16:47
zyuaneasy to add new tests16:47
maliniametts: what wud slow things down ?16:47
amettsSetting up and tearing down for each and every test in a suite.16:47
zyuanmore important: less time spend on find the reason of a very waired result16:47
cppcabreraametts: That setup/teardown is actually not very expensive. At the moment, the suite takes 10s to execute.16:47
zyuanametts: not every, some.16:47
cppcabreraAll 43 tests.16:48
zyuanametts: it's donw with cppcabrera's patch16:48
flaper87cppcabrera: but when it comes to functional tests it might be a bit slower16:48
amitgandhicppcabrera: 10 sec curently with or without teardown?16:48
cppcabrera10s w/ teardown + setup16:48
flaper87I mean, setUp marconi-server (db, whatsoever) run everything and then restart16:48
cppcabreraAgainst mongodb16:48
cppcabreraLive marconi16:48
amitgandhinice16:48
malini& the current test suite takes 13 sec ,let me add16:48
zyuanmalini: if you worry about it becomes hard to write complex & long test, you can call test in another test16:49
kgriffsmalini: so, currently, each method within the class is numbered to ensure they run in a certain order, correct?16:49
cppcabreraPerformance is not an issue, IMO.16:49
malinikgriffs: yes16:49
kgriffscppcabrera: +116:49
zyuancurrently yes16:49
cppcabreraMy argument is for helping long-term maintainability.16:49
Sriramyes they are ordered.16:49
kgriffslet's stay on topic16:49
kgriffsok16:49
kgriffsand in Robot they are ordered, right?16:49
maliniyes (& Robot is dead now :(  )16:50
maliniit's just nosetests16:50
kgriffsok, so this is the way you found to simulate what robot was doing16:50
maliniyes..16:50
kgriffsI'd like to just make an a-priori assumption here that order matters in tests, and malini needs to be able to enforce it somehow16:50
malini& really the numbering is just for the entire test suite setup16:50
maliniif 000 is the set up & 999 is the teardown for the suite16:51
kgriffsalso...16:51
flaper87git lgo16:51
flaper87ops16:51
*** demorris has quit IRC16:51
malinias long as those happen,  the tests are mostly covered16:51
flaper87is the order really necessary ?16:51
kgriffsthe downside is that adding a new test can kind of be a pain16:51
kgriffssince you have to shift the numbers16:51
zyuanand it's hard find the reason beased on the result16:52
cppcabrera000 can be replaced by setUpClass, and 999 can be replaced by tearDownClass - those are respected bu python test runners.16:52
malinikgriffs: no16:52
zyuanbecause the result depends on unrelated running of tests16:52
malinibut setuoClass & teardown is just at test level16:52
flaper87Why do we need that order ?16:52
malininot for the test suite16:52
kgriffsmalini: good point16:52
*** IlyaE has joined #openstack-meeting-alt16:52
cppcabreraNo. setUp is at test case level. setUpClass is at suite level. :)16:52
malinitht''s the only reason we have 000 & 99916:52
kgriffssetUpClass()16:52
malinicppcabrera: no, I tries it16:52
cppcabreraI verified that a few times working on my last project.16:52
* kgriffs learns something everyday16:52
*** sreshetnyak has joined #openstack-meeting-alt16:52
cppcabrera@classmethod setUpClass16:52
malinicppcabrea: tried*16:53
cppcabrera@classmethod tearDownClass()16:53
kgriffshmmm16:53
malinicppcabrera: IF you are right, we can get rid of the numbers in the suite16:53
malini'IF' ;)16:53
cppcabrera:D16:53
zyuanactually, i thing waht Robot does it to simulate a state machine16:54
malinilets not talk Robot..its goe16:54
malinigone*16:54
zyuanbut a sequencial numbering is just a linear action16:54
malinigone loooong back!!16:54
flaper87ok, 6 mins to go16:54
flaper87lets fire some actions16:54
cppcabreraYup, let's wrap up.16:54
flaper87:D16:54
zyuani suggest we to find some better appraoch to simulate a state machine16:54
zyuan(without using the numbers)16:54
maliniI am adding it to the bp16:55
flaper87So, what are the next steps w.r.t functional tests?16:55
cppcabreraSuggested actions: discuss patch further, develop user testing strategy, restructure tests16:55
maliniright now..FREEZE!16:55
flaper87Can we make them launch marconi-server and then make them run with tox ?16:55
* flaper87 freezes16:55
cppcabreraflaper87: Definitely.16:55
* flaper87 is frozen16:55
cppcabreraflaper87: Even launch a DB16:55
maliniwe have too many changes in the api at the moment & the next two weeks are critical for us to wrap things up.16:55
* cppcabrera defrosts flaper8716:56
amettsmalini: +116:56
flaper87cppcabrera: thaanks :D16:56
flaper87malini: good point16:56
*** sreshetnyak has quit IRC16:56
flaper87didn't think about that16:56
kgriffs+116:56
maliniI wont suggest making any radical changes right now, but need to add it to the bp & address it in two weeks16:56
*** ruhe has joined #openstack-meeting-alt16:56
maliniafter two weeks*16:56
kgriffs+ 1 zillion16:56
Sriram+116:56
flaper87functional tests are important for incubation, though!16:56
flaper87we have to have a good test suit16:56
maliniflaper87: is refactoring the tests important?16:57
flaper87malini: it is important running them with tox so they can run in the gate16:57
*** IlyaE has quit IRC16:57
maliniflaper87: ok.let's add tht to the things needed in 2 weeks16:57
flaper87and having them, obviously (which most probably means updating them with the latest API)16:58
flaper87cool16:58
maliniI can get it running in tox with the current tests (which is being updated for API changes)16:58
flaper87#action malini to make functional tests run with tox16:58
cppcabrera+116:58
kgriffsI think we can polish later. We will need to do some non-trivial tooling to get really elegant, robot-like behavior out of nose/unittest16:58
maliniI am going to add our discussion items to bp..we certainly have some great ideas to start hacking on in two weeks16:59
flaper87#action malini cppcabrera to update the etherpad with new strategies and changes for functional tests16:59
flaper87malini: and to the etherpad ^16:59
flaper87pls16:59
flaper87:D16:59
kgriffsbe sure to include a note about state machine for future discussion16:59
malininow lets talk placement :D16:59
cppcabreraAny lasSure thing. :)16:59
Sriram+20017:00
flaper87#action kgriffs to buy pop-tarts for the next summit17:00
flaper87erm, 1 min left17:00
malinikgriffs: sure17:00
cppcabrera**the typos^^17:00
kgriffs:D17:00
cppcabreraI think we're out of time. :P17:00
Sriramoh noes.17:00
flaper87AWESOME MEETING!!!!17:00
flaper87#endmeeting17:00
*** openstack changes topic to "OpenStack meetings (alternate)"17:00
openstackMeeting ended Fri Aug  2 17:00:50 2013 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:00
maliniyayy!!17:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-08-02-16.02.html17:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-08-02-16.02.txt17:00
cppcabreraYes. :D17:00
openstackLog:            http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-08-02-16.02.log.html17:00
Sriram:)17:01
*** zyuan has left #openstack-meeting-alt17:01
*** cppcabrera has left #openstack-meeting-alt17:01
*** amitgandhi has left #openstack-meeting-alt17:03
*** marcos-sb has quit IRC17:04
*** markmcclain has quit IRC17:06
*** markmcclain has joined #openstack-meeting-alt17:07
*** markmcclain has quit IRC17:09
*** IlyaE has joined #openstack-meeting-alt17:16
*** kgriffs is now known as kgriffs_afk17:19
*** vipul is now known as vipul-away17:24
*** vkmc has quit IRC17:25
*** megan_w has joined #openstack-meeting-alt17:25
*** megan_w has quit IRC17:26
*** ruhe has quit IRC17:26
*** megan_w has joined #openstack-meeting-alt17:26
*** esp has joined #openstack-meeting-alt17:26
*** NikitaKonovalov has quit IRC17:27
*** vipul-away is now known as vipul17:33
*** esp has left #openstack-meeting-alt17:33
*** NikitaKonovalov has joined #openstack-meeting-alt17:34
*** demorris has joined #openstack-meeting-alt17:37
*** NikitaKonovalov has quit IRC17:38
*** NikitaKonovalov has joined #openstack-meeting-alt17:41
*** NikitaKonovalov has quit IRC17:41
*** NikitaKonovalov has joined #openstack-meeting-alt17:43
*** ruhe has joined #openstack-meeting-alt17:46
*** kgriffs_afk is now known as kgriffs17:49
*** markmcclain has joined #openstack-meeting-alt17:54
*** basha has joined #openstack-meeting-alt18:00
*** ruhe has quit IRC18:01
*** kgriffs is now known as kgriffs_afk18:03
*** sarob has joined #openstack-meeting-alt18:08
*** sarob_ has joined #openstack-meeting-alt18:11
*** NikitaKonovalov has quit IRC18:12
*** sarob has quit IRC18:15
*** rnirmal has joined #openstack-meeting-alt18:21
*** SergeyLukjanov has joined #openstack-meeting-alt18:26
*** demorris_ has joined #openstack-meeting-alt18:27
*** demorris has quit IRC18:27
*** demorris_ is now known as demorris18:27
*** vipul is now known as vipul-away18:28
*** basha has quit IRC18:29
*** stanlagun has quit IRC18:40
*** kgriffs_afk is now known as kgriffs18:41
*** NikitaKonovalov has joined #openstack-meeting-alt18:41
*** kgriffs is now known as kgriffs_afk18:43
*** kgriffs_afk is now known as kgriffs18:44
*** benl_ has quit IRC18:46
*** NikitaKonovalov has quit IRC18:46
*** NikitaKonovalov has joined #openstack-meeting-alt18:47
*** akuznetsov has quit IRC18:49
*** NikitaKonovalov has quit IRC19:04
*** benl has joined #openstack-meeting-alt19:04
*** harlowja_ has joined #openstack-meeting-alt19:07
*** SergeyLukjanov has quit IRC19:14
*** sarob_ has quit IRC19:20
*** lastidiot has quit IRC19:20
*** bourke has quit IRC19:20
*** salv-orlando has quit IRC19:20
*** esheffield has quit IRC19:20
*** Ekaterina_F has quit IRC19:20
*** harlowja has quit IRC19:20
*** Guest29511 has quit IRC19:20
*** sarob has joined #openstack-meeting-alt19:21
*** salv-orlando has joined #openstack-meeting-alt19:23
*** joel has joined #openstack-meeting-alt19:23
*** joel is now known as Guest4823319:23
*** sarob has quit IRC19:25
*** bourke has joined #openstack-meeting-alt19:26
*** Ekaterina_F has joined #openstack-meeting-alt19:26
*** lastidiot has joined #openstack-meeting-alt19:27
*** benl has quit IRC19:27
*** esheffield has joined #openstack-meeting-alt19:29
*** malini is now known as malini_afk19:29
*** yidclare has quit IRC19:30
*** vipul-away is now known as vipul19:30
*** NikitaKonovalov has joined #openstack-meeting-alt19:30
*** megan_w has quit IRC19:33
*** ruhe has joined #openstack-meeting-alt19:41
*** whenry has quit IRC19:43
*** abaron has joined #openstack-meeting-alt19:46
*** benl has joined #openstack-meeting-alt19:58
*** benl has quit IRC20:06
*** ruhe has quit IRC20:13
*** NikitaKonovalov has quit IRC20:16
*** yidclare has joined #openstack-meeting-alt20:29
*** boris-42 has joined #openstack-meeting-alt20:30
*** tedross has quit IRC20:31
*** abaron has quit IRC20:31
*** benl has joined #openstack-meeting-alt20:32
*** benl has quit IRC20:36
*** jculp has quit IRC20:38
*** yidclare has left #openstack-meeting-alt20:48
*** hemanth_ has quit IRC20:57
*** esp has joined #openstack-meeting-alt21:02
*** esp has left #openstack-meeting-alt21:02
*** kgriffs is now known as kgriffs_afk21:11
*** NikitaKonovalov has joined #openstack-meeting-alt21:16
*** Sriram has quit IRC21:21
*** NikitaKonovalov has quit IRC21:21
*** dhellmann is now known as dhellmann_21:23
*** kgriffs_afk is now known as kgriffs21:24
*** boris-42 has quit IRC21:32
*** rnirmal has quit IRC21:33
*** zane has joined #openstack-meeting-alt21:38
*** boris-42 has joined #openstack-meeting-alt21:45
*** esheffield has quit IRC21:48
*** pdmars has quit IRC22:00
*** ametts has quit IRC22:01
*** demorris has quit IRC22:16
*** NikitaKonovalov has joined #openstack-meeting-alt22:17
*** NikitaKonovalov has quit IRC22:22
*** lastidiot has quit IRC22:33
*** Riddhi has quit IRC22:37
*** zane has quit IRC22:40
*** Riddhi has joined #openstack-meeting-alt22:58
*** sarob has joined #openstack-meeting-alt22:59
*** sarob has quit IRC23:10
*** sarob has joined #openstack-meeting-alt23:11
*** NikitaKonovalov has joined #openstack-meeting-alt23:17
*** sarob has quit IRC23:19
*** sarob_ has joined #openstack-meeting-alt23:20
*** markwash has quit IRC23:20
*** NikitaKonovalov has quit IRC23:22
*** IlyaE has quit IRC23:34
*** djohnstone has quit IRC23:36
*** tanisdl has quit IRC23:38
*** IlyaE has joined #openstack-meeting-alt23:39
*** salv-orlando has quit IRC23:39
*** IlyaE has quit IRC23:44
*** kgriffs is now known as kgriffs_afk23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!