Thursday, 2016-02-18

openstackgerritFei Long Wang proposed openstack/zaqar: Fix metadata update issue for v1.1 and v2  https://review.openstack.org/28094100:26
wanghaoflwang: ping00:41
wanghaoflwang: hi, I noticed this patch(https://review.openstack.org/#/c/276603/) merged fail since "Additional properties are not allowed (u'pool_group' was unexpected)", but zaqar server's change depends on this.00:45
wanghaoflwang: do you have any suggestion about this situation?00:45
*** rcernin has quit IRC00:46
flwangwanghao: yep, i saw that. and it's a little bit tricky00:56
flwangsince our server side is using the client functional test as gate job00:56
wanghaoflwang: yes, and client also need server to run gate job too....00:58
flwangwanghao: is it possible to resolve it on server side? like in Mitaka, we can accept both pool and pool_group00:58
flwangand totally remove the support of 'poo' name in Newton00:58
wanghaoflwang: well, this is a way I think we can try it.00:58
wanghaoflwang: sounds good.00:59
flwangthat can resolve current issue and we can also give end user some time to migrate00:59
wanghaoflwang: sure, will update the patch for this.00:59
flwangwanghao: i think you just need change a little bit at the api level01:00
wanghaoflwang: emm, so the client side depends on server side.01:01
flwangwanghao: for sure01:02
wanghaoflwang: okay :)01:02
*** achanda has quit IRC01:03
flwangwanghao: thanks for working on this01:06
wanghaoflwang: you're welcome :)01:17
*** achanda has joined #openstack-zaqar01:20
*** kgriffs is now known as kgriffs|afk01:43
*** wxy__ has quit IRC02:06
*** wxy has joined #openstack-zaqar02:10
*** godlike has quit IRC02:24
openstackgerritFei Long Wang proposed openstack/zaqar: Fix the TTL issue of subscriptions for MongoDB  https://review.openstack.org/27046402:26
*** godlike has joined #openstack-zaqar02:29
*** godlike has joined #openstack-zaqar02:29
*** david_cu has joined #openstack-zaqar03:15
*** flwang1 has quit IRC03:56
*** pt_15 has quit IRC05:55
*** GB21 has joined #openstack-zaqar06:26
wxyOur client's Jenkins check doesn't contain functional test which is included in Jenkins gate. Does anyone know why?06:40
wxythis one: gate-zaqarclient-dsvm-functional06:41
wxySo we can't get the functional error at once until send the patch to gate.06:42
*** achanda has quit IRC07:14
*** achanda has joined #openstack-zaqar07:17
*** GB21 has quit IRC07:40
*** rcernin has joined #openstack-zaqar07:50
*** pcaruana has joined #openstack-zaqar08:05
*** GB21 has joined #openstack-zaqar08:06
*** jtomasek|afk is now known as jtomasek08:15
*** dynarro has joined #openstack-zaqar08:29
*** achanda has quit IRC09:25
*** achanda has joined #openstack-zaqar09:28
*** achanda has quit IRC09:29
*** achanda has joined #openstack-zaqar09:46
*** openstackgerrit has quit IRC10:02
*** openstackgerrit has joined #openstack-zaqar10:03
*** achanda has quit IRC10:12
*** achanda has joined #openstack-zaqar10:14
*** achanda has quit IRC10:16
*** dynarro has quit IRC10:25
openstackgerritwanghao proposed openstack/zaqar: "group" word should be used instead "pool"  https://review.openstack.org/27393110:53
openstackgerritwanghao proposed openstack/zaqar: Update operation should return updated result  https://review.openstack.org/28056311:03
*** achanda has joined #openstack-zaqar11:17
*** mdnadeem has quit IRC11:22
*** achanda has quit IRC11:24
openstackgerritwanghao proposed openstack/zaqar: "group" word should be used instead "pool"  https://review.openstack.org/27393111:38
*** mdnadeem has joined #openstack-zaqar11:42
openstackgerritwanghao proposed openstack/python-zaqarclient: "pool_group" word should be used instead "pool"  https://review.openstack.org/27660312:14
*** GB21 has quit IRC12:14
openstackgerritwanghao proposed openstack/zaqar: Update operation should return updated result  https://review.openstack.org/28056312:28
EmilienMflwang: when you around, I need your help12:53
vkmcEmilienM, 'sup12:53
vkmc:P12:53
EmilienMhey12:53
EmilienMI'm trying to start zaqar12:54
EmilienMbut I got this http://logs.openstack.org/51/279351/23/check/gate-puppet-zaqar-puppet-beaker-rspec-dsvm-centos7/62fe4f5/logs/zaqar/zaqar.txt.gz12:54
EmilienMwhile I'm using replicaset12:54
EmilienMyou can see the conf http://logs.openstack.org/51/279351/23/check/gate-puppet-zaqar-puppet-beaker-rspec-dsvm-centos7/62fe4f5/logs/etc/zaqar/zaqar.conf.txt.gz12:54
openstackgerritwanghao proposed openstack/zaqar: "group" word should be used instead "pool"  https://review.openstack.org/27393112:55
vkmclet me see12:55
vkmcEmilienM, paste the conf file please12:55
vkmcsorry, you did12:56
EmilienMvkmc: you have it ^12:56
vkmcyeah12:56
EmilienMand the mongo RS is created : http://logs.openstack.org/51/279351/23/check/gate-puppet-zaqar-puppet-beaker-rspec-dsvm-centos7/62fe4f5/console.html#_2016-02-18_05_41_33_46312:56
EmilienM*before* starting zaqar12:56
EmilienMbut looking at zaqar/storage/mongodb/driver.py it seems I need more than 1 conn.node or a mongos12:57
EmilienMa mongos is a mongo shard?12:57
vkmcI see yeah... that should work12:58
vkmcmaybe it's a problem with the replica set12:58
EmilienMcan I just use uri = mongodb://127.0.0.1:27017 ?12:59
vkmcthe mongos is a service between the app and the mongodb instance12:59
EmilienMyeah, it's for sharding right?13:00
vkmcthat helps retrieving data from the shards13:00
vkmcyeah13:00
vkmcehm, changing the uri may work13:00
vkmctry that13:00
EmilienMok13:00
EmilienMvkmc: do you run mongodb in devstack?13:01
EmilienMyes you do, I see in plugin.sh13:01
EmilienMyou use mongodb://localhost:27017/zaqar for URL13:01
EmilienMwhat if I don't specify a dbname?13:02
vkmcEmilienM, it creates a default one13:08
vkmcEmilienM, you should be able to specify the uri without specifying the db for it13:09
vkmcEmilienM, are you working on the puppet modules for it?13:10
EmilienMvkmc: yes, puppet-zaqar is done13:15
EmilienMhttps://github.com/openstack/puppet-zaqar13:16
*** achanda has joined #openstack-zaqar13:22
EmilienMvkmc: still failing, logs are coming13:23
vkmcEmilienM, k k13:23
EmilienMI think I'll just try to reproduce on my vm and debug in there13:23
*** achanda has quit IRC13:27
therveEmilienM, Do you use replication or sharding?13:28
EmilienMtherve: just rs, but on a localhost13:28
EmilienMtherve: http://logs.openstack.org/51/279351/24/check/gate-puppet-zaqar-puppet-beaker-rspec-dsvm-centos7/378bd84/logs/13:29
EmilienMEither a replica set or a mongos is required to guarantee message delivery13:29
EmilienMI probably missed something... trying to reproduce13:30
therveYeah I guess it's an issue on how mongo is configured13:31
EmilienMit's weird, I created the Replicaset13:31
EmilienMhttp://logs.openstack.org/51/279351/23/check/gate-puppet-zaqar-puppet-beaker-rspec-dsvm-centos7/62fe4f5/console.html#_2016-02-18_05_41_33_46313:31
therveYeah that's just puppet output? What does it really do?13:32
therveEmilienM, http://logs.openstack.org/51/279351/24/check/gate-puppet-zaqar-puppet-beaker-rspec-dsvm-centos7/378bd84/logs/ps.txt.gz there is only one process here13:32
EmilienMit create a replicaset with https://github.com/puppetlabs/puppetlabs-mongodb/blob/master/lib/puppet/provider/mongodb_replset/mongo.rb13:32
EmilienMtherve: I don't run mongos, just RS so no extra process, isn't?13:33
therveEmilienM, You need several nodes in your set?13:33
EmilienMwhy?13:34
therveBecause that's what zaqar wants13:34
EmilienMI'm following http://docs.openstack.org/developer/zaqar/devref/development.environment.html13:34
EmilienMI run single node13:34
therveEmilienM, "This URI points to single mongodb node which of course is not reliable,"13:35
therveYou set unreliable to False13:35
therveSet it to true if you don't catre13:35
thervecare13:35
EmilienMok so if I want to run single node, I need to use unreliable13:35
therveOr run 2 mongo processes13:36
therveYou can run 2 processes on the a single node13:36
EmilienMI would like to save resources (memory)13:36
therveThen yeah set unreliable :)13:36
EmilienMtherve: look https://github.com/openstack/puppet-openstack-integration#description13:36
EmilienMwe already have lot of services13:36
EmilienMtherve, vkmc: thx for your help -- testing again with unreliable now13:39
*** GB21 has joined #openstack-zaqar13:43
vkmcEmilienM++13:43
*** amitgandhinz has joined #openstack-zaqar13:58
*** sriram has joined #openstack-zaqar14:09
EmilienMvkmc, therve: it worked! thanks again14:13
vkmcEmilienM, cool!14:14
*** dynarro has joined #openstack-zaqar14:32
*** jtomasek has quit IRC14:39
*** jtomasek has joined #openstack-zaqar14:44
EmilienMvkmc, therve: tempest fail though http://logs.openstack.org/15/281515/7/check/gate-puppet-openstack-integration-scenario002-tempest-dsvm-centos7/6e52338/console.html#_2016-02-18_14_26_43_958 -- but I think it's related to the logs I see in: http://logs.openstack.org/15/281515/7/check/gate-puppet-openstack-integration-scenario002-tempest-dsvm-centos7/6e52338/logs/zaqar/zaqar.txt.gz14:46
EmilienMStage <zaqar.storage.mongodb.messages.MessageQueueHandler object at 0x359fb90> does not implement create14:46
EmilienMshould I worry about that ^ ?14:46
therveNo I don't think that warning matters14:47
EmilienMonly some tests are failing: test_release_claim, test_delete_multiple_messages, test_get_multiple_messages and test_post_messages14:47
therveI don't think zaqar tempest tests are maintained though14:47
EmilienMshould they work with mongodb?14:48
EmilienMtherve: why tempest tests would not be maintained?14:48
EmilienMyou don't gate with tempest jobs?14:48
vkmcEmilienM, we don't14:49
EmilienMO_O14:49
therveYeah not anymore14:49
vkmcEmilienM, we don't have enough bandwidth to maintain those14:49
EmilienMvkmc: is it enough if I run api.messaging.test_queues ? (they all pass)14:50
EmilienMvkmc: http://logs.openstack.org/15/281515/7/check/gate-puppet-openstack-integration-scenario002-tempest-dsvm-centos7/6e52338/console.html#_2016-02-18_14_22_57_32514:50
therveIdeally you'd run the functional tests instead14:50
EmilienMtherve: well, you have to understand we run a single project for testing (tempest) and do go through each project to find how they run testing14:51
EmilienMwe basically run one command that test all projects: https://github.com/openstack/puppet-openstack-integration/blob/master/run_tests.sh#L10414:51
EmilienMI understand why you don't use tempest, you don't have enough people in your team14:53
EmilienMbut for people like us who like single interface to run functional tests, it's painful to have multiple tools to run testings14:53
EmilienMso we'll just run api.messaging.test_queues, I hope it's still useful14:54
therveNot using tempest is not a matter of people, rather something to do with being in control of our tests14:57
therveThat said, the tests there should be fixed14:57
ryansbEmilienM: also, there was a migration away from "all the anciliary projects use tempest"14:58
ryansbbecause it forced out-of-tree functional tests14:58
EmilienMtherve: no problem, I'm just giving feedback.14:59
EmilienMtherve: gnocchi also does not run tempest14:59
EmilienMaodh, same14:59
EmilienMetc14:59
EmilienMfor projects like puppet, were we have an integrated CI, we need a single tool to run our tests15:00
EmilienMand until now, tempest is good for us.15:00
therveEmilienM, So yeah that's why I don't get it. You need a single tool, except when you don't?15:01
therveWhat do you do for gnocchi?15:01
ryansbyeah, tempest is nice for you guys. Unfortunately it's tough for project maintainers since trees can be out of sync15:01
ryansband it was hard to get reviews on new tempest tests, which made the tests worse15:02
therveryansb, Does tempest-lib help with that? IE having tempest tests but in our tree?15:03
therveAlways wanted to check what it does precisely15:03
ryansbI don't know, actually. It's probably intended to address that case, but it's not something I've used yet15:05
EmilienMtherve: right, we try to run a single tool (tempest now) - for gnocchi, we don't test it directly, but we test ceilometer which use gnocchi dispatcher.15:07
EmilienMno problem, we'll just run api.messaging.test_queues15:07
Eva-iryansb: therve: hello, do you think it's a bug? https://bugs.launchpad.net/zaqar/+bug/154432815:17
openstackLaunchpad bug 1544328 in zaqar "Zaqar doesn't require X-PROJECT-ID header in requests (noauth)" [Undecided,New]15:17
*** GB21 has quit IRC15:19
*** kgriffs|afk is now known as kgriffs15:22
*** kgriffs is now known as kgriffs|afk15:22
ryansbI guess it depends if we want it to be a SHOULD or a MUST15:24
ryansbbased on our docs (must) then yeah, it's a bug15:24
*** achanda has joined #openstack-zaqar15:25
*** kgriffs|afk is now known as kgriffs15:25
*** achanda has quit IRC15:30
Eva-iryansb: thank you, Fei Long thinks it's a bug too. But no matter what our docs are saying, fixing it will break backward compatibility. Changes like this seem to be not considered as acceptable http://specs.openstack.org/openstack/api-wg/guidelines/evaluating_api_changes.html#guidance. Maybe it's acceptable for us, because we don't have much users, like Fei Long says. But if we continue breaking backward compatibility, we will never have much15:39
therveFrankly I don't even know why we have a noauth mode15:48
Eva-itherve: Maybe noauth can be useful for systems which are not exposed to the public where keystone isn't needed. I know that without keystone, Zaqar performs much faster.15:51
ryansbhm, tough spot then.15:57
*** zigo has quit IRC16:03
*** zigo has joined #openstack-zaqar16:05
*** pcaruana has quit IRC16:15
*** dynarro has quit IRC16:15
*** rcernin has quit IRC16:17
*** pt_15 has joined #openstack-zaqar16:43
Eva-iwow, this time I found really great bug https://bugs.launchpad.net/python-zaqarclient/+bug/154710016:45
openstackLaunchpad bug 1547100 in Python client library for Zaqar "Zaqar freezes on message POST request without Content-type header" [Undecided,New] - Assigned to Eva Balycheva (ubershy)16:45
Eva-iFeel free to confirm bugs you like https://bugs.launchpad.net/~ubershy16:52
*** exploreshaifali has joined #openstack-zaqar16:55
ryansbEva-i: cool, I'll plan to do some triage later today17:09
ryansbgot some heat stuff to work on first17:09
Eva-iryansb: oki =)17:10
*** achanda has joined #openstack-zaqar17:28
*** achanda has quit IRC17:33
*** rcernin has joined #openstack-zaqar17:36
*** achanda has joined #openstack-zaqar18:03
*** achanda has quit IRC18:26
*** achanda has joined #openstack-zaqar18:37
*** kgriffs is now known as kgriffs|afk18:56
*** achanda has quit IRC19:03
*** kgriffs|afk is now known as kgriffs19:08
*** achanda has joined #openstack-zaqar19:17
EmilienMhey zaqar folks, I reported https://bugs.launchpad.net/zaqar/+bug/154718819:35
openstackLaunchpad bug 1547188 in zaqar "some tempest tests are broken" [Undecided,New]19:35
*** exploreshaifali has quit IRC19:39
vkmcthanks EmilienM19:50
*** achanda has quit IRC19:51
*** rcernin has quit IRC19:55
*** achanda has joined #openstack-zaqar19:58
EmilienMvkmc: do you know why all zaqar tests are run while I'm trying to run only some of them on https://review.openstack.org/#/c/281515/10/run_tests.sh ?20:01
vkmcEmilienM, nope20:06
*** boris-42 has quit IRC20:16
*** boris-42 has joined #openstack-zaqar20:18
*** boris-42 has quit IRC20:24
EmilienMvkmc: I figured20:24
vkmcEmilienM, what's up?20:24
EmilienMall zaqar tests are tagged as smoke20:24
EmilienMand that's bad.20:24
EmilienMhttps://github.com/openstack/tempest/blob/master/tempest/api/messaging/test_claims.py#L5420:25
EmilienMyou tag all tests as smoke and tests are broken20:25
EmilienMso if someone (me here) runs smoke with zaqar enabled20:25
EmilienMtempest fails.20:25
EmilienMalso, it would be great if zaqar PTL could find a QA liaison: https://wiki.openstack.org/wiki/CrossProjectLiaisons#QA20:26
vkmcI see20:26
vkmcall right, we will check that out20:27
vkmcthanks EmilienM20:27
*** mtreinish has joined #openstack-zaqar20:30
EmilienMvkmc: https://review.openstack.org/28204820:30
vkmcEmilienM, that was quick, thx20:30
mtreinishEmilienM: that's probably a good segway for me to raise moving those tests into a tempest plugin20:31
mtreinishhttp://docs.openstack.org/developer/tempest/plugin.html20:32
EmilienMtrue, I'm concerned about all those projects not using tempest20:32
Eva-iflwang: hello, are you here?20:35
flwangEva-i: yep, reading the log20:36
flwangEmilienM: so test_messages and test_claim are still failing, right?20:37
EmilienMall the time20:37
EmilienMand only some of them20:37
EmilienMyou can see bug report in https://bugs.launchpad.net/zaqar/+bug/1547188/20:38
openstackLaunchpad bug 1547188 in zaqar "some tempest tests are broken" [Undecided,New]20:38
EmilienMflwang: also fyi https://review.openstack.org/28204820:38
flwangEmilienM: got, i will fix those failing tests asap20:40
EmilienMif you need help for testing let me know20:40
EmilienMI'm putting our CI system in place so it might help you20:41
flwangEmilienM: awesome, thanks20:41
flwangEmilienM: btw, why the puppet-zaqar has to depend on openstacklib?20:41
EmilienMflwang: but I tested puppet-zaqar and it works fine I guess, except if you see something wrong in configuration.20:41
EmilienMflwang: openstacklib is a puppet library we created, like Oslo in OpenStack20:41
EmilienMwe have common code in there20:42
flwangoh, ok20:42
Eva-iflwang: remember yesterday we talked about making Zaqar's error responses provide info in bodies? It's possible, however we'll be not able to make all 404 responses from Zaqar provide info - we can't control 404 responses when the user tries unexisting endpoints - this is handled by falcon library.20:42
flwangEmilienM: i got this comment from our operation " xavpaice: flwang1: that zaqar puppet module depends on puppet-openstacklib, which we currently haven't updated due to mass module breakage everywhere else"20:43
flwangEva-i: really? Even adding the 'description' for the error?20:43
EmilienMflwang: I suggest him/her to jump on IRC / ML and talk with us.20:44
flwangEmilienM: cool20:44
EmilienMflwang: "mass module breakage everywhere else" > usually because you forked our modules for your own needs and now you have hard time to catch-up with upstream.20:45
EmilienMI bet on that20:45
Eva-iflwang: we can add descriptions to the error responses in our endpoints code. If the user tries to make request to the endpoint which does not exist in Zaqar, falcon library automatically sends 404 response.20:48
Eva-iflwang: I mean this: http://paste.openstack.org/show/uOjpajzYIhhZIYhRvZ9C/20:53
flwangEmilienM: hah, i will forward the words ;)20:53
flwangEva-i: no20:54
flwangthat's not what we want to fix20:54
flwangor i would day it's totally different scenarios20:54
flwangthe problem we were discussing is a non-existing pool, flaovr and queue20:55
flwangnot a non-existing endpoint, see my point?20:55
*** flwang1 has joined #openstack-zaqar20:57
Eva-iflwang: I think I see your point. You would like to make 404 responses only decriptive for these situations. It's okay that Zaqar sometimes will produce 404 responses without descriptions, right?21:01
flwangEva-i: yes, and i think the later case you mentioned above, only happens when user access URL directly instead of using zaqar client21:02
Eva-iflwang: yeah21:02
Eva-iflwang: though you only mentioned non-existing pool, flavor and queue. But there are few more situations when python-zaqarclient can receive 404 response code. You can find all situations by searching "falcon.HTTPNotFound" and "falcon.HTTP_404" in our Zaqar repo.21:05
Eva-i*in zaqar.transport.wsgi21:06
flwangoh, yes, i can imagine21:07
*** amitgandhinz has quit IRC21:17
*** amitgandhinz has joined #openstack-zaqar21:18
openstackgerritFei Long Wang proposed openstack/zaqar: Fix metadata update issue for v1.1 and v2  https://review.openstack.org/28094121:20
flwangvkmc: ping21:24
vkmcflwang, pong21:26
flwangvkmc: https://review.openstack.org/#/c/280941/21:26
flwangi think you can start to work in the websocket part21:26
vkmcflwang, great!21:26
vkmcwill do that then21:26
flwangvkmc: awesome, thanks21:27
mtreinishflwang: did you see my comment about moving the zaqar tests into a tempest plugin? I'm trying to make sure we get tempest in-tree tests trimmed down to scope we defined in yvr before the end of the cycle21:28
mtreinishit'd be good to know who was owning it on the zaqar side (so I can pester them :) )21:29
flwangmtreinish: i don't think there is an owner, TBH21:32
flwangmtreinish: you can blame me :)21:33
flwangmtreinish: seriously, anything i can do to make everyone happy?21:33
mtreinishflwang: heh, no worries. I'm just raising it now really. I can help I'd just like to make sure it's on the project's radar21:34
mtreinishit should be quite simple to do the plugin migration, and I can help with it21:34
flwangmtreinish: awesome, and TBH, personally, i would like to migrate to use tempest as one of gate job21:36
flwangmtreinish: i'm a tempest fans21:36
flwangbut you know, we're a small team, we don't have much bandwidth21:36
flwangbut it's on my list of Newton21:37
mtreinishsure, I understand that21:37
flwangand if there is any volunteer from tempest side for zaqar, it would be awesome21:37
flwangotherwise, it will happen in Newton early stage21:38
*** sriram has quit IRC21:48
openstackgerritFei Long Wang proposed openstack/zaqar: Fix the TTL issue of subscriptions for MongoDB  https://review.openstack.org/27046421:49
*** amitgandhinz has quit IRC21:59
Eva-iflwang: from now on our Rally DSVM gate should pass. Rally team released fix for the bug I reported: https://review.openstack.org/#/c/281935/21:59
flwangEva-i: awesome, thanks for working on this22:03
openstackgerritMatthew Treinish proposed openstack/zaqar: Add zaqar tempest plugin  https://review.openstack.org/28207422:06
mtreinishflwang: ^^^22:06
mtreinishI haven't tested that at all or anything22:06
mtreinishbut if someone could take that to completion I'd appreciate it22:06
mtreinish(including the tempest removal patch once that's really ready)22:07
flwangmtreinish: cool, i will take over, thanks a lot22:08
flwangbut i will surely bug you for questions22:08
mtreinishheh, sure22:09
Eva-iflwang: how do you think, is it possible to run python-zaqarclient CLI commands with noauth?22:13
Eva-iflwang: I mean to make the client not get auth token from keystone.22:14
flwangi don't really understand the question22:29
flwangwhy do you want to do that?22:29
flwangIMHO, noauth is the lowest priority22:30
flwangi don't think any user want to use that22:30
Eva-iflwang: I wonder if I can get 401 response from Zaqar when using CLI22:30
Eva-iflwang: okay22:30
flwangEva-i: so what's the background?22:31
Eva-iflwang: I just need to decide should I make python-zaqarclient's transport exceptions provide default error messages, so the CLI user will see the error even if Zaqar doesn't provide description in error response, nevermind22:38
Eva-iflwang: seems like no need to make default error messages22:39
*** jtomasek has quit IRC23:05
*** openstackgerrit has quit IRC23:47
*** openstackgerrit_ is now known as openstackgerrit23:47
*** openstackgerrit_ has joined #openstack-zaqar23:47
*** openstackgerrit_ is now known as openstackgerrit23:48
*** openstackgerrit_ has joined #openstack-zaqar23:48
*** openstackgerrit_ has quit IRC23:55
*** openstackgerrit_ has joined #openstack-zaqar23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!