07:00:16 #startmeeting swift 07:00:17 Meeting started Wed Jun 14 07:00:16 2017 UTC and is due to finish in 60 minutes. The chair is acoles. Information about MeetBot at http://wiki.debian.org/MeetBot. 07:00:18 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 07:00:20 The meeting name has been set to 'swift' 07:00:33 Hi who is here for the swift meeting? 07:00:38 o/ 07:00:39 o/ 07:00:43 hello! 07:00:47 o/ 07:00:48 o/ 07:00:50 hello 07:01:01 o/ 07:01:05 o/ 07:02:17 good morning/afternoon/evening 07:02:44 hi 07:02:52 good morning 07:02:58 Good afternoon! 07:02:59 Thanks for attending this meeting, the agenda is here ... 07:03:00 #link https://wiki.openstack.org/wiki/Meetings/Swift 07:03:20 note that the agenda for this meeting (0700) is in the right hand column 07:03:29 welcome to the second other meeting :) 07:03:58 notmyname doesn't want it to be called anything other than "Swift meeting" ;) 07:04:18 mattoliverau: yes, this is the second in our new 0700UTC meeting slot 07:04:20 and I second that 07:04:27 oic, like that 07:04:36 mahatic: yes, this is the "0700 swift meeting" 07:04:55 shame we can't call it the 007 meeting ;) 07:05:03 or can we? ;) 07:05:17 lol 07:05:18 Rather than duplicating the agenda for the 2100 meeting I tried to select some topics that would involve people likely to be here at this time 07:05:20 lol, looks like it 07:05:30 mattoliverau: that sounds secret 07:05:32 acoles: nice 07:05:42 kota_: lol, top secret 07:05:45 mattoliverau: call it what you like, just showing up is great :) 07:05:52 acoles: yup, looks good 07:06:12 so to start with there are some pieces of information to share... 07:06:25 #topic Denver PTG 07:06:37 * acoles is learning the meeting bot commands :) 07:07:09 in case you missed the announcement, the PTG in Denver now has dates and a hotel block confirmed 07:07:18 #link http://lists.openstack.org/pipermail/openstack-dev/2017-June/118002.html 07:07:40 week of 11th - 15th September 07:07:51 cool 07:08:10 this is the 'dedicated developer gathering', so swift will have 3 full days of meeting room allocated 07:08:23 on Weds-Thurs 07:08:33 tho I don't think I'll be there, not going to self fund cause I'm unemployed.. and if I do happen to have a new job by then, it might be too late (or early) to ask for travekl :( 07:08:36 thanks for the reminder 07:08:59 Mon & Tues will be for 'cross project' or 'inter project' work 07:09:00 though it's unlikely that I'd attend 07:09:08 mattoliverau: :( 07:09:14 mahatic: :( too! 07:09:32 yeah, mahatic and I will just do our own one... virtually :) 07:09:41 I know, too bad! 07:09:50 mattoliverau: it's a deal! ;) 07:09:52 the PTG should IMHO be more productive than the forum in Boston due to us having more dedicated meeting times 07:10:03 +1 07:10:10 I expects that 07:10:18 +1 I'd expect that as well 07:10:19 the last PTG was great 07:10:23 there is a tentative room schedule here tentative room allocation plan 07:10:23 #link https://docs.google.com/spreadsheets/d/1xmOdT6uZ5XqViActr5sBOaz_mEgjKSCY7NEWcAEcT-A/edit#gid=397241312 07:10:24 so if you can be there, try to be :) 07:11:00 so, for those who may be able to attend, start making plans 07:11:31 any questions on the ptg? 07:11:59 not at this point 07:12:01 ok 07:12:05 #topic Sydney summit call for presentations 07:12:30 now thiis one I hope I can at least self fund.. so come to my home country :) 07:12:33 briefly, the Sydney summit (November) call for presentations is open 07:12:47 :) mattoliverau nice! 07:12:49 #link https://www.openstack.org/summit/sydney-2017/call-for-presentations/ 07:13:06 mattoliverau: come to your home?? 07:13:16 sure! 07:13:17 :D 07:13:26 oh , country! (line wrapped for me after home!) 07:13:32 come visit the beac house :( 07:13:34 :) 07:13:48 moving on... 07:13:52 #topic deprecating known-bad EC policy 07:14:21 you should be aware of patch 468105 07:14:34 patchbot? bah, you let me down! 07:14:39 lol 07:14:44 #link https://review.openstack.org/#/c/468105/ 07:14:48 lol 07:14:53 and notmyname also send a mailing list about it 07:14:58 yeah 07:14:59 #link http://lists.openstack.org/pipermail/openstack-dev/2017-June/118242.html 07:15:04 notmyname promised me patchbot would be here to hold my hand :) 07:15:11 mattoliverau: thanks 07:15:28 and that email has also gone to the ops ML 07:15:31 #link http://lists.openstack.org/pipermail/openstack-operators/2017-June/013735.html 07:15:43 oh yeah 07:16:02 we have discussed this in previous meetings, but it's worth bringing up again, in case any one has questions/concerns 07:16:38 in summary, once that patch lands, swift services will not start if there are known bad polices *that have not been deprecated* 07:17:33 so that is likely to be in the next release of Swift 07:18:24 if you or your customers are using the known-bad isa-l-rs-vand configuration then they MUST deprecate it and really should be moving data away from it 07:18:43 any questions? 07:18:57 okay. Looks like the change could use some eyes, unless we're deliberately waiting to merge it 07:19:54 mahatic: that is a good point, it might be one of those patches that we like to collect several +2 to be sure that the community as a whole has adopted the change 07:19:56 acoles: Is it documented? 07:20:28 psachin: that's also a great question! 07:20:40 acoles: yeah, sounds good 07:20:44 I don't see any doc changes in that patch https://review.openstack.org/#/c/468105/ 07:21:03 we have been alerting the bad policies for a while, so maybe there is an existing doc warning 07:21:34 shouldn't the "UpgradeImpact" tag in commit msg trigger some doc changes? 07:21:53 nope I do not yet find any doc 07:22:24 ok, well there's a potential review comment that could be left on gerrit 07:22:47 i think it had deprication warning in the changelog 07:22:59 i thought 07:23:03 mahatic: no I don't think anything automatic happens as a result of UpgradeImpact - the tag is there for users to grep and find likely ugrade issues 07:23:13 and we log the warning, I also thohught 07:23:24 perhaps, it's good to add the info to doc/source/overview_erasure_code.rst? 07:23:44 yeah, thats where I was just looking but couldn't see antthing 07:23:45 mattoliverau: right, it is in the changelog but it does feel like the EC docs should say 'don't use this config' 07:23:52 acoles: ah okay 07:23:54 * kota_ is searching the word about that 07:23:55 +100 07:23:56 maybe they do and I don't see it 07:24:06 kota_: +1 07:24:24 would someone volunteer to leave a comment on the pacth re docs? 07:24:53 I'll do it 07:24:59 mahatic: thanks 07:25:08 mahatic: thank you 07:25:17 mahatic: +1 acoles +1 07:25:20 psachin: thanks for raising that issue 07:25:34 ok, let's move on 07:25:46 #topic doc changes 07:26:01 continuing the docs theme... 07:26:16 This is a heads-up for forthcoming changes in the way openstack docs are maintained. Previously the bulk of 07:26:16 docs was maintained by the docs team, but with changing resources they have decided to move more of the docs 07:26:16 over to individual projects. So project teams will be taking more responsibility for maintaining docs. 07:26:35 * acoles prepared that statement earlier :) 07:27:02 here's the openstack spec for the changes 07:27:03 #link https://review.openstack.org/#/c/472275/ 07:27:28 lol, thumbs up for the prep 07:27:32 summary - we, swift team, will be maintaining more docs in the future 07:27:53 swift already has install guide and api ref in swift repo, as well as the longer standing docs tree 07:28:14 yeah most the doc team where laid off with me :( 07:28:21 which kind of docs will be in? 07:28:25 but we will also get the admin guide and swiftclient will get the CLI reference, if I understand things correctly 07:28:29 as a new ones 07:28:37 oic 07:29:04 there is also a quite lengthy etherpad detailing the implications for each project... 07:29:04 #link https://etherpad.openstack.org/p/doc-migration-tracking 07:29:10 kota_: listed there ^^ 07:29:24 thx 07:29:27 AFAICT part of the plan will be to unify the various docs under a single doc/ dir 07:29:44 I expect that when this move happens we may discover that some of the docs we inherit will need some updating 07:30:09 now that would be awesome. 1 doc tree not a million 07:30:12 so that's a heads-up - it has not happened yet but it is coming 07:30:15 mattoliverau: agree 07:31:21 ok, tovin07_ added a topic to agenda so I want to skip to that before we get short on time 07:31:23 #topic osprofiler in swift 07:31:38 yes 07:31:49 thanks acoles 07:31:51 tovin07_: what's up? can you tell us a little about the patch? 07:31:52 #link https://review.openstack.org/#/c/468316/ 07:32:26 I'll try to integrate osprofiler into swift 07:32:48 osprofiler is an distributed tracing lib in openstack 07:33:12 it's in nova, cinder, glance, keystone, neutron, heat,.. already 07:33:49 the key thing here is that osprofiler can trace request across openstack services 07:34:14 such as a request come from nova -> keystone -> glance -> cinder .... 07:34:22 tovin07_: is it using request id for that? 07:34:42 nope 07:35:55 an example in magnum 07:35:57 #link https://tovin07.github.io/magnum/cluster-create-with-50-iteration-limit.html 07:36:28 it traces across magnum, keystone, nova, heat 07:36:52 and traces http, rpc and db calls 07:37:00 my brief understang is for osprofiler is that it is for making performance analytics inside of component, right? 07:37:33 kota_, yes, it's can be used to trouble-shooting as well 07:38:21 and at the fist time, i'm feeling swift already has x_profile middelware for such a purpose. 07:38:35 recently, I discussed with notmyname here http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift.2017-05-03.log.html#t2017-05-03T16:05:17 07:38:46 i'm at natural yet 07:39:19 yes, I see, however, x-profile works with swift only 07:39:32 tovin07_: yes 07:39:53 so that sounds like swift logger vs oslo logging tome 07:39:55 to me 07:40:15 osprofiler is one of openstack "official"(?) analytics tool 07:40:21 I see the patch has a link to the openstack spec for osprofiler 07:40:22 but swift has already that one 07:40:27 #link https://review.openstack.org/#/c/103825/ 07:40:47 tovin07_: osprofiler only traces requests or it can be used to trace background task like object replication? 07:41:01 tovin07_: has the patch changed since the irc conversation with notmyname , in terms of dependencies? 07:41:17 kota_, yes, https://github.com/openstack/osprofiler 07:42:42 acoles, yes, we only added osprofiler as new dependency, no oslo.config... 07:43:07 and we only added osprofiler into test-requirements 07:43:07 psachin, currently, it traces requests (db, http, rpc), we also have plan to make osprofiler traces background tasks and periodic tasks as well 07:44:23 its interesting, how can we test it? do I need to use devstack or something? 07:44:34 +1 07:44:47 can it grow other swift metrics? ie, maybe intergrate x-profier into it 07:44:49 tovin07_: what is different about the swift middleware vs middleware for other projects? just curious that there isn't a single generic middleware for all services? 07:45:03 tovin07_: Thats great. Thanks. 07:45:14 mattoliverau, just enable it via swift configuration file and use OSC cli along with --profile option 07:45:31 cool 07:46:16 hieulq: but presumably there is some infrastructure needed for osprofiler to gather/publish metrics? 07:46:17 acoles: I assume it needs to intergrate into project specific things, like our db calls etc 07:46:43 acoles, i don't quite understand your question, can you make it clearer 07:46:51 acoles, default is rabbitmq, but you can use ceilometer, redis, mongodb, ELK.. 07:47:02 hieulq, tovin07_ : swift can also output statsd, I wonder if we can get performance metrix out via there too. 07:47:38 tovin07_: it may be a stupid question ;) but for example with keystone we just include keystonemiddleware same as every other project, but here we have a swift specific middleware implementation 07:47:52 acoles, here is the list of backend drivers now: https://github.com/openstack/osprofiler/tree/master/osprofiler/drivers 07:48:01 hieulq: neither of those are used in swift in default 07:48:31 hieulq: thanks 07:48:35 mattoliverau, yeah, we may do it in next cycles 07:49:08 acoles, oh yes, because of dependencies --> I have to "custom" osprofiler middleware a little bits for swift 07:49:26 tovin07_: got it, makes sense 07:50:03 kota_, yes, but if users used swift as back-end or along with other projects, we have rabbitmq as default 07:50:21 tovin07_: so what do you need at this point in time? is the patch ready for reviewing? 07:50:51 and in case we use swift stand-alone, at least users need redis/mongo/etc to store the traces 07:50:54 acoles, yes, it is ready for reviewing now 07:51:45 ok community, take note and pls review if you can ^^ 07:52:01 thanks :) 07:52:40 ok.. so I don't have redis or mongo.. so thatll be an extra stumbling block for me. but i'll try and loop round to it when I get some time. 07:52:43 acoles, thanks :D 07:52:59 if only there was a swift driver :P 07:53:13 tovin07_: it will help reviewers if there is a doc somewhere showing how to set up the other infrastructure (rabbitmq or whatever is simplest) - not all swift devs are familar with installing other openstack services since swift often runs standalone (or with only keystone) 07:53:37 mattoliverau: +1 07:53:49 acoles: yup, +100 07:53:52 mattoliverau: it seeems to make us easy to test in functional or probe 07:54:02 kota_: yeah 07:54:03 acoles, yes, for simplest case, you can test it with redis or mongodb 07:54:17 ok, time is passing - thank you tovin07_ and hieulq for coming along to discuss osprofiler 07:54:32 yeah thanks hieulq and tovin07_ 07:54:37 #topic priority reviews 07:54:43 thanks guys 07:55:01 very quickly, do we have any updates on progress on these patches... 07:55:18 Make eventlet.tpool's thread count configurable in... 07:55:18 #link https://review.openstack.org/#/c/289664/ 07:55:39 I've pushed up a new version. but still doing a bit of testing as I was getting the wrong behaviour on my SAIOs.. 07:55:42 mattoliverau: cschwede did you get chance to look at that? 07:55:50 mattoliverau: oh, great, thanks! 07:56:05 mattoliverau: nice 07:56:22 This one we discussed at last 0700 meeting has merged ... ssync failing to sync expired object 07:56:22 #link https://review.openstack.org/#/c/456921/ 07:56:33 but rledisez is fixing another ssync bug 07:56:49 #link https://review.openstack.org/472659 07:56:50 which is for bug 07:56:50 #link https://launchpad.net/bugs/1652323 07:56:51 Launchpad bug 1652323 in OpenStack Object Storage (swift) "ssync syncs an expired object as a tombstone" [Medium,Confirmed] 07:56:54 yep. i have the bugfix, i need to write tests 07:57:00 rledisez: do you need reviews/help? 07:57:08 rledisez: ok :) tests! 07:57:19 rledisez: I'll try to help with that 07:57:35 better to wait i wrote tests i think, but the patch is really small so if you want to have a look, it's good to see if i'm in the right direction 07:57:45 If I have some time later this week, I'll loop round and try and take a look.. but can't promise atm. 07:57:51 rledisez: name_lookup/domain_remap bugs - any more bugs to fix there? 07:58:09 one to come, but not pushed yet. and also i"'ll work on func test soon 07:58:10 rledisez: ack I will wait for you to push another version of patch - first glance it looked ok 07:58:25 acoles: ok, good to know, thx 07:58:44 ring part power increase 07:58:44 #link https://review.openstack.org/#/c/337297/ 07:58:44 any progress? 07:58:48 cschwede: ^^ ? 07:59:13 we should just merge the damn tihing ;) 07:59:16 looks like cschwede is not around 07:59:41 last i heard tim was going back and forward on it, but that was quite a whle ago 07:59:46 ok, NM, well time is up anyway 07:59:55 #topic next meeting 07:59:55 Next 0700UTC meeting will be on June 28th 07:59:55 Next 2100UTC meeting is today 07:59:58 thx acoles for chairing :) 08:00:05 +1 08:00:11 rledisez: thanks! phew 08:00:14 thanks acoles 08:00:17 Good night rledisez 08:00:23 thank you for coming and working on swift 08:00:29 #endmeeting