Wednesday, 2011-12-21

*** debo-os has quit IRC00:05
*** dragondm has quit IRC00:07
*** _adjohn has joined #openstack-meeting00:08
*** adjohn has quit IRC00:11
*** _adjohn is now known as adjohn00:11
*** sleepsonthefloo has quit IRC00:22
*** edgarmagana has quit IRC00:23
*** jog0 has joined #openstack-meeting00:24
*** jog0 has left #openstack-meeting00:24
*** scottsanchez has quit IRC00:36
*** adjohn has quit IRC00:39
*** adjohn has joined #openstack-meeting00:39
*** Kiall has quit IRC00:39
*** scottsanchez has joined #openstack-meeting00:48
*** Kiall has joined #openstack-meeting00:48
*** salv has quit IRC01:06
*** zns has joined #openstack-meeting01:06
*** Ravikumar_hp has quit IRC01:06
*** sleepsonthefloo has joined #openstack-meeting01:15
*** lloydde has quit IRC01:16
*** lloydde has joined #openstack-meeting01:31
*** jdurgin has quit IRC01:33
*** ricky_99 has joined #openstack-meeting01:41
*** Gollen has joined #openstack-meeting01:41
*** Rick_Che has joined #openstack-meeting01:44
Rick_Chehello everyone01:45
Gollenhello rick.01:45
ricky_99meeting is over?01:49
*** thinbottle has joined #openstack-meeting01:51
*** lloydde has quit IRC01:56
*** file_ has joined #openstack-meeting01:59
*** file_ has left #openstack-meeting02:00
*** Gollen is now known as rick_zhu02:03
*** rick_zhu is now known as Gollen02:06
*** CDY has quit IRC02:08
*** lloydde has joined #openstack-meeting02:09
*** lloydde has quit IRC02:16
*** debo-os has joined #openstack-meeting02:18
*** debo-os has quit IRC02:23
*** adjohn has quit IRC02:37
*** bengrue has quit IRC02:49
*** lloydde has joined #openstack-meeting02:54
*** lloydde has quit IRC02:59
*** gyee has quit IRC03:00
*** oubiwann has quit IRC03:29
*** mahmoh has joined #openstack-meeting03:47
*** adjohn has joined #openstack-meeting03:48
*** adjohn has quit IRC03:52
*** lloydde has joined #openstack-meeting03:58
*** jakedahn has quit IRC04:05
*** dmd171 has joined #openstack-meeting05:10
*** dmd171 has left #openstack-meeting05:10
*** lloydde has quit IRC05:14
*** Rick_Che has quit IRC05:35
*** lloydde has joined #openstack-meeting05:45
*** Ravikumar_hp has joined #openstack-meeting05:53
*** sleepsonthefloo has quit IRC06:11
*** dolphm has joined #openstack-meeting06:23
*** lloydde has quit IRC06:40
*** ricky_99 has quit IRC06:51
*** Ravikumar_hp has quit IRC06:59
*** dolphm has quit IRC07:06
*** thinbottle has quit IRC07:07
*** deshantm has quit IRC07:45
*** adjohn has joined #openstack-meeting07:53
*** adjohn has quit IRC08:17
*** Gollen has quit IRC09:46
*** derekh has joined #openstack-meeting10:02
*** darraghb has joined #openstack-meeting10:40
*** zns has quit IRC10:41
*** dprince has joined #openstack-meeting13:50
*** ayoung-hanukah is now known as ayoung13:55
*** dolphm has joined #openstack-meeting14:19
*** bcwaldon has joined #openstack-meeting14:20
*** Ravikumar_hp has joined #openstack-meeting14:27
*** dolphm has quit IRC14:47
*** jsavak has joined #openstack-meeting14:57
*** joesavak has joined #openstack-meeting15:06
*** jsavak has quit IRC15:09
*** jsavak has joined #openstack-meeting15:10
*** joesavak has quit IRC15:12
*** Ravikumar_hp has quit IRC15:35
*** rnirmal has joined #openstack-meeting15:37
*** dolphm has joined #openstack-meeting15:38
*** dwalleck has joined #openstack-meeting15:43
*** deshantm has joined #openstack-meeting15:43
*** CDY has joined #openstack-meeting15:52
*** dragondm has joined #openstack-meeting16:00
*** zns has joined #openstack-meeting16:05
*** dolphm has quit IRC16:07
*** dolphm has joined #openstack-meeting16:10
*** zns has quit IRC16:34
*** Ravikumar_hp has joined #openstack-meeting16:37
*** Ravikumar_hp has quit IRC16:41
*** mattray has joined #openstack-meeting16:41
*** Ravikumar_hp has joined #openstack-meeting16:42
*** dolphm has quit IRC16:44
*** dolphm has joined #openstack-meeting16:49
*** donaldngo_hp has joined #openstack-meeting16:52
*** nati2 has joined #openstack-meeting17:00
nati2Hi Folks!17:01
*** rnirmal has quit IRC17:01
*** AntoniHP has joined #openstack-meeting17:01
dwalleckHi!17:01
AntoniHPhello17:01
donaldngo_hphello17:02
dwalleckHmm, no jaypipes?17:02
Ravikumar_hpjaypipes - vacation17:03
dwalleckAhhh17:03
dwalleckI'll be doing the same next week17:03
AntoniHPI think it would make sense to have not meeting next week17:03
dwalleckI agree. There will be little activity till after the new year17:04
Ravikumar_hpyes.17:04
dwalleckDo we still have enough to discuss for today?17:04
*** donaldngo_hp has quit IRC17:04
*** donaldngo_hp has joined #openstack-meeting17:04
dwalleckI sent out the example test changes that we had talked about before. I know Jay's been busy so I figured I'd give him a hand17:04
*** ElJose_ has joined #openstack-meeting17:05
nati2Thanks!17:05
nati2OK let's get started17:05
dwalleckLets see if this works...17:05
*** longshot has joined #openstack-meeting17:05
dwalleck#startmeeting17:05
openstackMeeting started Wed Dec 21 17:05:35 2011 UTC.  The chair is dwalleck. Information about MeetBot at http://wiki.debian.org/MeetBot.17:05
openstackUseful Commands: #action #agreed #help #info #idea #link #topic.17:05
*** rohitk has joined #openstack-meeting17:05
dwalleckNice!17:05
dwalleck#topic Project status17:05
nati2#topic example tests17:05
*** openstack changes topic to "Project status"17:05
notmynamebtw, ElJose_ and longshot are rackspace cloud files (ie swift) qa people17:06
dwalleckerr17:06
*** sleepsonthefloo has joined #openstack-meeting17:06
dwalleck#topic example test17:06
*** openstack changes topic to "example test"17:06
nati2Ah ok! Let's talk about  Project status first :)17:06
dwalleckSo did that email make it out? I saw some weird bounced response17:06
AntoniHPwhat was subject?17:06
dwalleckI was also on a pile of cold medicine last night, so I'm hoping things were clear :)17:06
rohitki read it17:07
rohitkand I liked your proposal17:07
dwalleckWe'll stick with with the example test cases17:07
*** zns has joined #openstack-meeting17:07
dwalleckAnd the discussion of test class structure going forward17:07
nati2Gotca!17:07
Ravikumar_hpdwalleck: i do not see that email17:08
AntoniHPI have not seen it too, did it go to openstack general list?17:08
rohitkdwalleck: however there is a one issue17:08
dwalleckDid that sound reasonable? nati2, AntoniHP, I was specifically aiming for an approach that met your goals17:08
dwalleckIt went to openstack-qa-team17:09
donaldngo_hpi got it. it was in my junk email box17:09
donaldngo_hpcheck there guys17:09
dwalleckrohtik: What issue did you find?17:09
Ravikumar_hpyes. it is junk email box . found it17:09
rohitka test server was created for ServerTests in setup() and we are kind of pushed to use that server throughout the test methods in that class17:10
rohitkwhat if I want to create more test servers with different params17:10
nati2 dwalleck: I understand this is point " Tests do not change the expected state of the resources under test"17:10
nati2dwalleck: Then we do setup in setup method17:11
dwalleckrohitk: Correct. But I thought that was the point that people wanted was that they wanted to make many assertions on one action, and see each of those results17:11
rohitkof course it's just an example that you have stated, I just think that it reduces the flexibility to write various combinations in a single class17:11
rohitkdwalleck: yes, as per the last meeting17:12
dwalleckSo the one thing I forgot to mention was that I could only see this being used for primary test cases for something like create server17:12
rohitkdwalleck: yes, so the top level positive primary tests17:12
dwalleckIf you want to do some specific tests, you would do all of that within one test method17:12
dwalleckBut if you want to setup one scenario and test many, many things about it, or know that you will expand testing of it later, this something like this might be a good approach17:13
* AntoniHP is reading list archive17:13
rohitkdwalleck: Also I would not be able to run delete_server tests before the __verify_tests, as they would fail17:13
dwalleckrohitk: Correct. I did note that delete server would needed to be tested elsewhere17:14
rohitkdwalleck: It's a better than previous approach to have class level fixture though17:14
dwalleckrohitk: I agree, class level fixtures are the ideal situation17:15
dwalleckThat will help when we move to parallel execution17:15
rohitkdwalleck: +117:15
dwalleckSo, what I would like to suggest is to see some examples from any interested person of how they would think this would work best17:16
nati2Let's me confirmed delete server test case?17:16
dwalleckThis is just my opinion, and I'd like to see many more :)17:16
AntoniHPit makes sense, however why call to create server is in setup rather then a test?17:16
nati2dwalleck: If I wanna test delete server (test_delete_server), we should create a server inside test_delete_server ?17:17
*** joesavak has joined #openstack-meeting17:17
dwalleckAntoniHP: To allow for parallel execution of the tests. If the server is already created, and all that must be done is validations on the create response, the state of the server, or for example ping17:18
rohitkAntoniHP: Check the previous pastebins you can understand the difference in the existing and proposed, IMP is an improvisation17:18
dwalleckThen all this can be done at once17:18
rohitk*IMO17:18
dwallecknati2: Correct. You would write a single test method for a case like that17:18
nati2dwalleck: I got it17:19
dwallecknati2: Or, it could be a test class if you want to do many assertions about deletion of the server17:19
AntoniHPbut I think the idea would be that parallel execution would run  signle test class, then only thing that would benefit from parallel execution is creation and deletion of server17:20
dwalleckfor example, delete the server in setup, save the response in the setup, and the tests could be test response code, test if you get a 404 for a GET for that server, etc17:20
nati2dwalleck: How about prohibit overwrite of values? like this http://pastebin.com/WRa3GqTS17:20
*** jsavak has quit IRC17:20
nati2dwalleck: "and the tests could be test response code," I like this idea!! dwalleck++17:21
dwallecknati2: You could certainly do that. I really hope I wouldn't have to write defensive code in my tests, but it's certainly possible17:21
nati2dwalleck:  I think to we should have method to check this policy17:22
nati2dwalleck: Some test method may break parallel execution17:22
dwalleckAntoniHP: I'm not completely sure I follow. If I think I understand, what you're saying is that parallel execution would still have to wait for the class setup to run, and that's right17:22
dwalleckThat's the trade off. But then you only have to wait for one server to build, not 5. It's an improvement17:23
AntoniHPdwalleck: I think a signle process should execute a single class and one class should never be split among many processes17:23
dwallecknati2: I see your point...sorry, Python isn't my primary coding language, but don't they have some concept like Java's "final" keyword?17:24
dwalleckAntoniHP: So what you're saying if that I have 10 test classes, I would spawn 10 test threads, each executing one class?17:25
rohitk dwalleck: we use that in config.py17:25
rohitkwhere we define methods as properties17:25
AntoniHPdwalleck: no more than 10, but yes, no class methods should be executed in parallel17:25
nati2dwalleck: like this https://github.com/openstack/tempest/blob/master/tempest/config.py#L2317:25
dwalleckrohitk: That has the same effect, but I was hoping it could be as simple as "final image_ref..."17:26
dwalleckAntoniHP: Ahh, now I understand. Why would you not want each test class running at the same time?17:27
rohitknati2++17:27
donaldngo_hpso we will have class names like test_create_delete_server.py in antonis case. in darryls case we will have test_server.py and have test_delete_server method inside along with other methods to test server api. both can be ran in parallel17:27
AntoniHPdwalleck: if there are 100s of scenarios then running them all at once could be a bottleneck17:27
donaldngo_hpwe just have more classes17:27
nati2dwalleck: It says no final keyword http://stackoverflow.com/questions/802578/final-keyword-equivalent-for-variables-in-python17:27
dwallecknati2: Ugh, weird Python =P17:28
nati2dwalleck: But we can write read only class17:28
AntoniHPit is finally I think17:28
dwallecknati2: Right17:28
AntoniHPalso python does not have real multithreading, and with multiprocessing class code would be executed multiple times anyway17:29
dwalleckAntoniHP: I think I fell off the wagon.  Could you tell me what you think a test class should look like and do?17:30
*** rohitk has quit IRC17:31
AntoniHPhttp://pastebin.com/WRa3GqTS]17:31
AntoniHPhttp://pastebin.com/WRa3GqTS17:31
dwalleckAntoniHP: One of my teammates I believe has an improved multithreaded execution method. I still need to look at it17:32
AntoniHPnosetests uses multiprocessing17:32
dwalleckAntoniHP: And then you want each test to run in order always, correct?17:33
AntoniHPI would like each test within a class to be exectued in order17:33
AntoniHPhowever each class, containing a sequence of tests, be able to execute in parallel17:34
*** donaldngo_hp has quit IRC17:34
dwalleckAntoniHP: Okay, now I'm clear17:34
*** donaldngo_hp has joined #openstack-meeting17:34
*** lloydde has joined #openstack-meeting17:35
dwalleckAre you not concerned about the implicit dependencies that creates?17:35
*** lloydde_ has joined #openstack-meeting17:35
*** lloydde has quit IRC17:35
dwalleckI'm trying to understand how creating ordered tests helps. To me that complicates the tests by quite a bit17:36
AntoniHPto be clear lets call test something that is test, and ntest something that is test in nosetest software17:37
AntoniHPso I would like test to be convention a class containing ntests17:37
dwalleckThe way I've always handled things before is to have no test depend an anything other that data it retrieves or is created in a fixture17:37
AntoniHPthe reason behind it is it helps with reporting by increasing resolution of data that nosetests return back17:38
AntoniHPit is also much easier to follow by humans (IMO), and in larger teams this is of high importance17:39
dwalleckHmm, then the last example I provided did not meet it's goal. The purpose of that example was to meet the desire to have more clear reporting17:40
dwalleckSo for the result of one test, you would like to see the results of many assertions, correct?17:41
AntoniHPI would see it as ntest_1 - make REST call to create server and store reponse somewhere, if call failed mark it17:41
AntoniHPntest_2 verify reponse or if no response skip test17:41
AntoniHPetc17:41
*** derekh has quit IRC17:42
dwalleckSo I think the difference we have is the granularity we want from results of our tests17:42
dwalleckOr even the granularity of our tests17:42
AntoniHPthat is right17:43
dwalleckSo what you're saying is that create server is a test, which has many test cases, which will all be part of one test class17:43
AntoniHPI propose to disccuss it more on the list, rather then take more time on this meeting17:43
dwalleckIs that right?17:43
donaldngo_hpAntoni isnt your approach like how Kong was?17:43
dwalleckI agree17:43
dwalleckI don't think we're going to solve this today17:43
donaldngo_hpi like Kong btw17:43
AntoniHPok, I will write some emails on during Christmas, I was following wrong mainiling list until today :-(17:44
dwalleckBecause I really don't follow this approach. But lets keep discussing this. I'm sure we can come to some type of middle ground17:45
dwalleckYes, I think lots of code examples and even result examples would be great!17:45
dwalleckAnd this will give us all some time to mull things over17:46
dwalleckMost of my team is already on vacation, and I know they'll want to chime in as well17:46
AntoniHPagree17:47
dwalleckOkay, so we'll table this discussion for the moment17:47
dwalleck#topic General discussion17:47
*** openstack changes topic to "General discussion"17:47
dwalleckRight now I don't have much else to report. I'm trying to make sure whatever work my team is doing is updated in Launchpad17:47
dwalleckDoes anyone else have any other topics they want to bring up?17:49
dwalleckGoing once?17:49
dwalleckTwice?17:49
AntoniHPMerry Christmas :-) Happy New Year17:50
dwalleckhave a great holiday!17:50
dwalleck#endmeeting17:50
*** openstack changes topic to "Openstack Meetings: http://wiki.openstack.org/Meetings | Minutes: http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/"17:50
openstackMeeting ended Wed Dec 21 17:50:10 2011 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:50
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-12-21-17.05.html17:50
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-12-21-17.05.txt17:50
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-12-21-17.05.log.html17:50
nati2Happy holiday!17:50
*** dwalleck has quit IRC17:50
*** nati2 has quit IRC18:07
*** nati2 has joined #openstack-meeting18:08
*** reed has quit IRC18:11
*** jog0 has joined #openstack-meeting18:13
*** jog0 has left #openstack-meeting18:13
*** anotherjesse has joined #openstack-meeting18:14
*** dolphm_ has joined #openstack-meeting18:25
*** dolphm has quit IRC18:28
*** hggdh has quit IRC18:33
*** donaldngo_hp has quit IRC18:33
*** oubiwann has joined #openstack-meeting18:34
*** rohitk has joined #openstack-meeting18:36
*** dolphm_ has quit IRC18:37
*** dolphm has joined #openstack-meeting18:37
*** hggdh has joined #openstack-meeting18:42
*** dolphm has quit IRC18:43
*** martines has quit IRC18:43
*** deshantm has quit IRC18:45
*** bengrue has joined #openstack-meeting18:53
*** darraghb has quit IRC18:59
*** dragondm has quit IRC19:04
*** zns1 has joined #openstack-meeting19:09
*** zns has quit IRC19:11
*** donaldngo_hp has joined #openstack-meeting19:15
*** bcwaldon has left #openstack-meeting19:20
*** donaldngo_hp has quit IRC19:24
*** novas0x2a|laptop has quit IRC19:26
*** novas0x2a|laptop has joined #openstack-meeting19:26
*** nati2_ has joined #openstack-meeting19:28
*** nati2 has quit IRC19:28
*** dragondm has joined #openstack-meeting19:29
*** rohitk has quit IRC19:31
*** dragondm has joined #openstack-meeting19:34
*** jsavak has joined #openstack-meeting19:35
*** dragondm has quit IRC19:37
*** nati2_ has quit IRC19:38
*** joesavak has quit IRC19:39
*** hggdh has quit IRC19:49
*** zns1 has quit IRC20:00
*** longshot has quit IRC20:02
*** jsavak has quit IRC20:04
*** zns has joined #openstack-meeting20:07
*** nati2 has joined #openstack-meeting20:10
*** oubiwann has quit IRC20:11
*** joesavak has joined #openstack-meeting20:15
*** dolphm has joined #openstack-meeting20:16
*** oubiwann has joined #openstack-meeting20:19
*** nati2_ has joined #openstack-meeting20:24
*** nati2 has quit IRC20:25
*** hggdh has joined #openstack-meeting20:27
*** Ravikumar_hp has quit IRC20:38
*** dprince has quit IRC20:46
*** Ravikumar_hp has joined #openstack-meeting20:47
*** jakedahn has joined #openstack-meeting20:47
*** sandywalsh has quit IRC20:57
*** zns has quit IRC20:59
*** zns has joined #openstack-meeting21:00
*** jsavak has joined #openstack-meeting21:03
*** joesavak has quit IRC21:03
*** anotherjesse has quit IRC21:24
*** anotherjesse has joined #openstack-meeting21:25
*** bhall_ has quit IRC21:42
*** bhall has joined #openstack-meeting21:45
*** rohitk has joined #openstack-meeting21:49
*** jsavak has quit IRC21:50
*** rohitk has quit IRC22:09
*** rohitk has joined #openstack-meeting22:09
*** dolphm has quit IRC22:35
*** rnirmal has joined #openstack-meeting22:35
*** rohitk has quit IRC22:38
*** HowardRoark has joined #openstack-meeting22:42
*** HowardRoark has quit IRC23:03
*** GheRivero_ has joined #openstack-meeting23:04
*** nati2_ has quit IRC23:18
*** nati2 has joined #openstack-meeting23:19
*** GheRivero_ has quit IRC23:22
*** nati2 has quit IRC23:31
*** rnirmal has quit IRC23:45
*** dolphm has joined #openstack-meeting23:46
*** mattray has quit IRC23:47
*** dolphm has quit IRC23:50

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!