Monday, 2015-04-20

*** km___ has joined #openstack-swift00:48
*** km has quit IRC00:49
*** kei_yama has quit IRC00:50
*** kei_yama has joined #openstack-swift00:51
*** ChanServ sets mode: +v peluse01:27
*** remix_tj has quit IRC01:52
*** remix_tj has joined #openstack-swift01:52
*** Vinsh has joined #openstack-swift01:56
*** remix_tj has quit IRC01:57
*** Vinsh has quit IRC02:00
*** proteusguy has quit IRC02:37
notmynamehello mandarine02:55
*** david-lyle has quit IRC02:55
notmynamealso, mattoliverau02:58
*** jamielennox is now known as jamielennox|away02:58
mattoliveraunotmyname: hey hey02:59
notmynamemattoliverau: in my morning tomorrow, I want to get a handle on the essential patches for RC2. any input you have is invaluable03:02
mattoliveraunotmyname: k, I'll do a review of our reviews and send you my suggested additions :)03:03
*** jamielennox|away is now known as jamielennox03:03
notmyname:-)03:04
notmynamethanks03:04
notmynameI just updated https://wiki.openstack.org/wiki/Swift/PriorityReviews with a section03:07
notmynamethere's a section for RC203:07
notmynamemattoliverau: so if you have stuff to add (or remove), that's the place03:07
notmynamestuff for an RC2 needs to be stuff that fixes a big issue in RC1 and already has a patch to master03:07
*** silor has joined #openstack-swift03:13
*** kei_yama has quit IRC03:14
mattoliverauk03:27
*** km___ has quit IRC03:36
*** kei_yama has joined #openstack-swift03:37
pelusenotmyname, I don't see the patch from clayg on the ECrecon that goes with the new pyeclib (he posted a github version of it last week).  Maybe you should ping him tomorrow about that one and add it to the list.03:40
peluseI'll be off/on all week (technically on vacation)03:40
*** kei_yama has quit IRC03:54
*** kei_yama has joined #openstack-swift03:58
*** Vinsh has joined #openstack-swift04:06
*** Vinsh has quit IRC04:13
*** kota_ has joined #openstack-swift04:53
notmynamekota_: hello!05:33
notmynamekota_: I'm about to go to bed, but tomorrow I'll be organizing the RC2 patches. anything from you to take advantage of pyeclib 1.0.7 needs to be submitted asap (I'll also be talking to clayg about it tomorrow)05:34
kota_nomyname: hello :)05:34
kota_notmyname: yup, pyeclib 1.0.7 thing for global requirement seems to be blocked, right?05:36
notmynamekota_: not for technical reasons afaik. I think we'll be able to get it in05:37
notmynameat least I'm hoping until I'm told otherwise)05:37
notmynamekota_: for now, assume it will land.05:38
kota_notmyname: ok I'll look at the status again.05:39
kota_notmyname: thanks to care about that!05:39
notmyname:-)05:40
notmynamehave a good rest of your day. I'm going to bed05:40
kota_notmyname: have a nice sleep, good night ;)05:41
*** zhill has joined #openstack-swift05:54
*** kota_ has quit IRC06:03
*** SkyRocknRoll has joined #openstack-swift06:14
*** SkyRocknRoll has joined #openstack-swift06:14
*** kei_yama has quit IRC06:19
*** kei_yama has joined #openstack-swift06:19
*** zhill has quit IRC06:37
*** jamielennox is now known as jamielennox|away06:52
*** Vinsh has joined #openstack-swift07:13
*** remix_tj has joined #openstack-swift07:15
*** Vinsh has quit IRC07:18
*** geaaru has joined #openstack-swift07:27
*** km_ has joined #openstack-swift07:29
*** chlong has quit IRC07:29
*** bkopilov has quit IRC07:32
*** bkopilov has joined #openstack-swift07:32
openstackgerritChristian Schwede proposed openstack/swift-bench: Fix bug when using --log-level option  https://review.openstack.org/17528507:36
openstackgerritChristian Schwede proposed openstack/swift-bench: Fix bug when using --log-level option  https://review.openstack.org/17528507:38
acolesnotmyname: re patch 174875 , not sure re RC2, probably clayg's call based on whether he believes there is serious enough failure scenario to backport07:44
patchbotacoles: https://review.openstack.org/#/c/174875/07:44
*** jistr has joined #openstack-swift07:47
*** jordanP has joined #openstack-swift08:04
openstackgerritKota Tsuyuzaki proposed openstack/swift: Bump PyECLib version from 1.0.3 to 1.0.7  https://review.openstack.org/17422508:23
openstackgerritKota Tsuyuzaki proposed openstack/swift: Use reconstruct insetad of decode/encode  https://review.openstack.org/17529908:24
*** kota_ has joined #openstack-swift08:25
*** chlong has joined #openstack-swift08:50
*** proteusguy has joined #openstack-swift08:52
*** foexle has joined #openstack-swift09:13
*** chlong has quit IRC09:20
*** pcaruana has quit IRC09:27
*** pcaruana has joined #openstack-swift09:30
*** chlong has joined #openstack-swift09:32
openstackgerritKota Tsuyuzaki proposed openstack/swift: Validates the number of data/parity on EC  https://review.openstack.org/17532509:46
*** silor has quit IRC09:46
openstackgerritKota Tsuyuzaki proposed openstack/swift: Validates the number of data/parity on EC  https://review.openstack.org/17532509:48
*** aix has joined #openstack-swift10:01
*** haomaiwa_ has quit IRC10:03
*** haomaiwa_ has joined #openstack-swift10:04
*** haomaiwa_ has quit IRC10:09
*** haomaiwang has joined #openstack-swift10:18
straycatDoes anyone know if there's any particular reason this guide http://docs.openstack.org/juno/install-guide/install/apt/content/swift-install-storage-node.html suggests mounting the xfs partitions with nobarrier, from what I've read in the man page this doesn't seem particularly safe10:53
portante_straycat: isn't barriers for when you are working with shared storeage?11:00
portante_Are you sharing disks between swift nodes?11:00
straycatportante_, i don't know, i will do some reading, but no i'm not sharing storage devices11:04
straycatportante_, ahh no it seems drives can reorder writes held in their cache11:06
*** silor has joined #openstack-swift11:14
straycathttps://lwn.net/Articles/283161/ explains it nicely11:20
*** astellwa1 has joined #openstack-swift11:23
*** astellwa1 has quit IRC11:23
*** km_ has quit IRC11:29
*** kota_ has quit IRC11:35
*** acoles is now known as acoles_away11:48
*** kei_yama has quit IRC11:53
*** jistr has quit IRC11:57
*** chlong has quit IRC11:58
*** haomaiwang has quit IRC12:01
*** jistr has joined #openstack-swift12:03
*** dbay has joined #openstack-swift12:10
*** dbay has quit IRC12:10
*** acoles_away is now known as acoles12:11
portante_straycat: you might want to read XFS related info: http://xfs.org/index.php/XFS_FAQ#Q:_How_can_I_address_the_problem_with_the_disk_write_cache.3F12:17
*** dencaval has joined #openstack-swift12:24
*** jamielennox|away is now known as jamielennox12:31
*** jamielennox is now known as jamielennox|away12:32
*** erlon has joined #openstack-swift12:34
*** esker has quit IRC12:52
*** bkopilov has quit IRC13:27
*** krykowski has joined #openstack-swift13:32
*** bkopilov has joined #openstack-swift13:35
*** annegentle has joined #openstack-swift13:35
*** jistr_ has joined #openstack-swift13:35
acolestsg: you around?13:37
*** jistr has quit IRC13:38
*** esker has joined #openstack-swift13:40
*** bkopilov has quit IRC13:40
*** bkopilov has joined #openstack-swift13:42
*** jistr_ has quit IRC13:44
*** jistr has joined #openstack-swift13:44
*** bkopilov has quit IRC13:49
*** SkyRocknRoll has quit IRC13:52
*** imkarrer has joined #openstack-swift14:01
*** Vinsh has joined #openstack-swift14:04
*** bkopilov has joined #openstack-swift14:04
*** Vinsh has quit IRC14:06
*** minwoob_ has joined #openstack-swift14:11
*** admin6 has joined #openstack-swift14:17
*** Vinsh has joined #openstack-swift14:20
*** Vinsh_ has joined #openstack-swift14:22
*** pberis has joined #openstack-swift14:24
*** Vinsh has quit IRC14:25
*** timburke has quit IRC14:25
*** timburke has joined #openstack-swift14:26
*** Vinsh_ has quit IRC14:26
*** winggundamth has joined #openstack-swift14:28
winggundamthhi. anyone know about rsyncd service in Swift? I want to know what exactly it is doing and how port 873 is important to this?14:31
*** jrichli has joined #openstack-swift14:33
ctenniswinggundamth: if object data needs to be copied from one machine to another, rsync is used.  The rsyncd daemon listens on port 873 for those connections.14:33
winggundamthis it including container and account sync as well?14:34
ctennisyes14:34
winggundamthif I change rsyncd daemon listen port. Which config that I need to change?14:34
*** acoles is now known as acoles_away14:36
winggundamthand what happen if no rsyncd run?14:37
ctennisI'm not sure where to change the port value, and if it doesn't run then your data will not be able to copy around when you make storage changes or have drive failures.14:38
winggundamthso rsync will important when only storage changed like remove or add new storage right?14:41
winggundamthnot for normal object, container, account operation14:42
*** mahatic has joined #openstack-swift14:43
*** vinsh has joined #openstack-swift14:46
*** acoles_away is now known as acoles14:47
*** annegentle has quit IRC14:57
*** annegentle has joined #openstack-swift14:59
*** pberis has quit IRC15:02
*** annegentle has quit IRC15:04
*** annegentle has joined #openstack-swift15:05
*** annegentle has quit IRC15:06
*** annegentle has joined #openstack-swift15:10
*** jistr has quit IRC15:14
jordanPhi guys. I know that an object can have a forward slash '/' in its name to support "pseudo folder". My question is, is it possible/valid to have two consecutive forward slash in an object name ?15:16
notmynamejordanP: yes15:16
notmynamejordanP: /v1/AUTH_foo/barcontainer/x/////y   <-- totally valid path in swift15:17
jordanPnotmyname, thanks !15:17
winggundamthjust wonder if python -c 'from swift.common.utils import whataremyips; print whataremyips()' return15:18
winggundamth['127.0.0.1', '::1', '10.10.10.1', 'fe80::74c1:1ff:fe0d:521b', '172.17.0.122', 'fe80::42:acff:fe11:7a']15:18
*** annegentle has quit IRC15:18
winggundamthwhich ip that Swift will use?15:18
notmynamewinggundamth: all of them15:21
notmynamewinggundamth: https://github.com/openstack/swift/blob/master/swift/common/ring/utils.py#L23315:22
*** minwoob_ has quit IRC15:22
notmynamewinggundamth: the my_ips parameter is normally the output of whataremyips15:22
*** krykowski has quit IRC15:23
winggundamthso what is dev_ip and dev_port? is it specify in .conf file?15:25
notmynamewinggundamth: those would be from the ring15:25
notmynamewinggundamth: the idea most of the time is in replication to see if the target location (according to the ring) is local or remote15:25
notmynamewinggundamth: eg don't try to sync to a remote location. like in https://github.com/openstack/swift/blob/master/swift/obj/replicator.py#L46715:27
notmynamewinggundamth: the ips parameter there is the output of whataremyips()15:27
*** foexle has quit IRC15:27
notmynamewinggundamth: that method is building a list of stuff that needs to be replicated to some other place in the cluster15:27
notmynamewinggundamth: so it filters out anything that's "local"15:28
*** jistr has joined #openstack-swift15:28
notmynamejordanP: I gotta be careful! I say something in IRC and then see it quoted 5 minutes later in an update email from github ;-)15:28
winggundamthnotmyname: ok. trying to understand what happen if the ring is public ip but node itself is private ip that doing NAT right now15:30
notmynamewinggundamth: that sounds...unusual15:31
winggundamthnotmyname: so it will fail on this method right https://github.com/openstack/swift/blob/master/swift/common/ring/utils.py#L23315:31
*** lpabon has joined #openstack-swift15:31
winggundamthYes I know. Trying to understand so I can fix this15:32
notmynameacoles: re https://review.openstack.org/#/c/174875/ the idea (as i understand from clayg) is that we probably don't ever want multiple FAs on one drive because that _reduces_ durability (less disperse across failure domains)15:32
winggundamthCan I create another private ip interface on each node that can connect together and use that?15:33
winggundamthlike proxy and nodes having private ip to connect each other and the way to go outside is only proxy15:33
notmynamewinggundamth: the idea is that all of the storage nodes need to be routable to each other15:34
jordanPnotmyname, yup ! You are the source of truth :)15:34
hurricanerixgood morning15:34
notmynamewinggundamth: and yes, that's very common. the proxy is the only thing that should have "public" access (ie outside the cluster)15:34
notmynamehurricanerix: hello!15:34
winggundamthhurricanerix: 22:34 here :)15:34
winggundamthnotmyname: ok. I'll try that way. So I need to config the ring with only private ip right?15:35
acolesnotmyname: thx i left a comment there earlier, i guess i'm confused why we did all the work to support it15:36
notmynamewinggundamth: yes. and all of the IPs in it need to be routable to each other (and of course the proxy servers)15:36
notmynameacoles: oops?15:36
notmyname;-)15:36
acolesnotmyname: maybe. and like we never enabled it to ever happen :/15:37
notmynameyeah15:37
notmynameclayg was fixing it and then realized the fix would probably be worse than the problem. and the problem probably isn't actually a problem15:37
notmynameacoles: re pyeclib 1.0.7. the thing is that the issue is in liberasurecode. BUT, on pypi liberasurecode is bundled with pyeclib. so therefore, bumping pyeclib1.0.7 is what's required. and since you've installed them independently you see the difference15:38
acolesnotmyname: yup. i think i just about caught up with that today. oh well!!15:38
acolesnotmyname: so why doesn't pip install pyeclib==1.0.7 also install the latest liberasurecode?15:39
notmynameI thought it did15:39
acolesnot for me15:40
acolesit made noise but didn't install, but then i have been very confused today :)15:40
notmynamethis is why golang's "static link everything into one binary" might be the best thing about the language ;-)15:41
notmynameacoles: mostly it's at this point I make disparaging comments about packages and dependencies and ask tsg to tell me what's really going on ;-)15:41
acolesnotmyname: :)15:42
notmynameacoles: ok, now I'm trying to make sense of patches15:42
acolesnotmyname: i have to leave early, kid stuff, have a good day15:43
notmynameacoles: https://review.openstack.org/#/c/173973/ vs https://review.openstack.org/#/c/17413815:43
notmynameacoles: ah, ok15:43
notmynamei'll get it sorted later today15:43
acolesi hope i haven't confused things!15:43
notmynameno worries :-)15:43
*** acoles is now known as acoles_away15:44
* notmyname thinks, "I'll just log in while I drink coffee and check on what's going on." Hours later...."I should probably actually get dressed and go to the office"15:44
*** gyee has joined #openstack-swift15:45
*** jistr_ has joined #openstack-swift15:53
egonnotmyname: that's often my morning.15:54
notmyname:-)15:55
egonactually, that's my morning right now.15:55
notmynameif everyone else wouldn't get up so many hours before me, I wouldn't have this problem!15:55
notmynamewhat's with these "time zones"?15:55
notmynameI guess it's my fault for living on the western end of the global time zones ;-)15:56
*** jistr has quit IRC15:56
egontrue, plus, it's not like SF gets up early anyway.15:58
notmynameI have small kids. I get up early no matter what SF culture says ;-)15:59
egonLOL. truth.15:59
*** jistr_ is now known as jistr16:05
*** minwoob has joined #openstack-swift16:07
*** wbhuber has joined #openstack-swift16:15
winggundamthnotmyname: thanks for you suggestion. now I can fix my problem :)16:16
*** winggundamth has quit IRC16:16
notmynameok, I've talked to ttx and have a good idea on the process needed for RC2 stuff. that's my priority today (ie done byt eh time I go to sleep)16:19
notmynamemy goal is to have a good list of stuff that needs to be in RC2 on https://wiki.openstack.org/wiki/Swift/PriorityReviews (and landed ASAP, today preferably)16:20
notmynamethe reguirements change has been approved for pyeclib 1.0.7 in global requirements for master and stable/kilo, so work can progress there16:20
notmynametime to get ready for the day and go to the office16:21
claygmorning16:24
*** jistr has quit IRC16:24
notmynameclayg: hi. see above for RC2 patches. acoles_away left a note on the priority reviews page with a question. and the other stuff needs to be tracked there16:26
claygacoles_away: I think we justified the work often because of the percieved need for a disk to be able to hold two indexes without loosing data16:26
claygbut I'm not sure if we did anything we didn't need to - fi in datapath tracking suffixes hashes per fi16:27
claygif we ever found we want to use rsync for revert we'll be very glad that having mutli fi's is supported16:28
notmynameok, now I'm really, really going to not be online16:29
claygacoles_away: I had the same issue with liberasurecode - I think something that was supposed to check liberasure version in pyeclib doesn't work - it wasn't an issue when I uninstalled everything and reinstalled newest pyeclib 1.0.7 from scratch16:29
imkarrerGood afternoon, I am looking for some help with integrating Ceilometer with Swift.  Does anyone have experience setting up the middleware which is responsible for the notifications?  Which yum package contains the middleware for swift?  I would prefer not to get the source from github as indicated in the Manual.  I have set up an SAIO with Ceilometer using the source from github, but I would like to use yum packages instead.16:37
*** zaitcev has joined #openstack-swift16:40
*** ChanServ sets mode: +v zaitcev16:40
claygzaitcev: ^16:40
*** admin6 has left #openstack-swift16:41
zaitcevclayg: Sorry, but I joined too late to see the message you're pointing at.16:41
imkarrerclayg: were you pointing to my message or your message to acoles concerning liberasure?16:42
claygzaitcev: imkarrer wasn't to use yum packages on a saio to test out ceilometer16:42
zaitcevwasn't? so he has become now? what?16:43
imkarrerzaitcev:  I am trying to setup ceilometer with swift for production.  I would prefer to install it via yum instead of cloning from github.16:44
zaitcevYou're IRCing from an iPhone today, aren't you :-)16:44
claygzaitcev: yeah that was typo s/wasn't/wants16:44
claygit's just not enough coffee16:44
zaitcevimkarrer: Okay, that makes sense. So, what is the issue? It's not one of those permissions thing, is it?16:44
imkarrerzaitcev: I figured out the permissions stuff by cloning via git, oh what fun.  I am trying to install the source via yum, but there are several ceilometer packages and I have yet to find the one which provides the middleware as an egg.16:46
zaitcevThe most common problem I see is that Ceilometer has a separate user ID (with a name like "ceilometer"). But its middleware runs in Swift under "swift". So, people run yum install openstack-ceilometer, edit the pipeline, and then everything looks nice, but can't work.16:46
zaitcevWait, what? Install "the source" via yum?16:46
imkarrerzaitcev: Maybe I am mis-using terminology.  I need to install the middelware via yum and not clone from git.  You indicate that openstack-ceilometer should provide the middelware.  Will try more with that package.16:47
zaitcevhold on, let me look16:49
zaitcevActually, I have no idea about the details of Ceilomoter. I'll need to rely on you doing something like  "rpm -ql openstack-ceilometer| grep middleware"16:50
imkarrerzaitcev: the command you request returns nothing.  The packages I have to choose from are openstack-ceilometer-{alarm, api, central, collector, common, compute}  let me try that command on the common package16:52
*** haomaiwang has joined #openstack-swift16:55
imkarrerzaitcev, openstack-ceilometer-common appears to give me the egg i need.  Thanks for the insight.  Hitting some distribution not found issues, ill will manually install the missing packages, hopefully there are no version conflicts with swift :D16:56
imkarrerzaitcev: I must run to a meeting, thanks again for your help today.16:57
zaitcevnp16:57
*** thumpba has joined #openstack-swift16:58
*** thumpba has quit IRC16:58
openstackgerritClay Gerrard proposed openstack/swift: Use reconstruct insetad of decode/encode  https://review.openstack.org/17529917:04
*** thumpba has joined #openstack-swift17:05
*** thumpba_ has joined #openstack-swift17:06
*** thumpba has quit IRC17:10
notmynamegood morning again17:33
*** zhill has joined #openstack-swift17:40
*** lcurtis has joined #openstack-swift17:46
*** geaaru has quit IRC17:48
*** jordanP has quit IRC17:50
claygnotmyname: so I finished going over all of acoles_away notes' on the 409/revert-to-handoff patches - and I think he'd agree we're probably better off doing nothing in RC2 than either of the suggested patches18:01
*** annegentle has joined #openstack-swift18:01
claygnotmyname: I think there's something I can pull out of patch 175076 that might still be useful - i'm going to work on that18:02
patchbotclayg: https://review.openstack.org/#/c/175076/18:02
claygthe fix for revert will probably end up being a combination of acoles_away work and some additional stuff to avoid fragment index duplication when it's reasonable to do so18:03
clayg*maybe* for now we could make the early and loud failure mode on revert when the primary node index is unavailable a little more on purpose than a unhnadled KeyError in RC2, but meh.18:05
claygI think with the etag pop in place the only other need-to-have is the pyeclib/reconstruct stuff!18:05
claygacoles_away and I will continue to debate the "ideal error handling when dealing with failures while recovering from failure" on master unless we find something far worse than we have so far18:08
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873618:10
claygthe whole thing reminding me of the scene from some star trek episode about the *secondary backup* - found the transcript -> https://gist.githubusercontent.com/clayg/3f84c2563754433b4247/raw/183498fdb2dae9b09a59f5e4c80a4aa80a211438/gistfile1.txt18:14
torgomaticbelt, suspenders, duct tape18:15
*** tsg has joined #openstack-swift18:17
tsgnotmyname, looks like we are close to master merge for the pyeclib update to 1.0.7 .. will go ahead and submit a review for stable/kilo18:21
tsg(unless someone else has already submitted the same - I didn't see one)18:22
claygi'm heading in18:22
claygtsg: i didn't see it on the wiki18:22
tsgclayg: ok - will submit the stable/kilo review now18:23
claygtsg: oh wait - yeah I did - https://review.openstack.org/#/c/174171/18:23
claygoh that's the rquirements repo?18:23
tsgyea .. don't have one yet for swift18:23
tsghave one only for swift master18:23
claygtsg: is this it?  https://review.openstack.org/#/c/174225/318:24
claygidk, you'll have to ask notmyname18:24
tsgclayg: for whatever reason, my topic got set to "stable/kilo" :) but that one is actually for master18:24
claygtsg: patch 175299 is being tracked on the prioirty reviews page (https://wiki.openstack.org/wiki/Swift/PriorityReviews)18:25
patchbotclayg: https://review.openstack.org/#/c/175299/18:25
claygso as long as that gets merged to stable/kilo it should al workout18:25
tsgyep.  clayg: https://review.openstack.org/#/c/175525/18:26
imkarrerzaitcev:  So installing the ceilometer middleware from openstack-ceilometer-common (Juno) leads to package resolution and conflict issues.  First it cannot find pysnmp.  After installing that there is conflict with oslo.rootwrap.  After upgrading rootwrap, no matter what i do (uninstall, rm -rf, pip install, easy_install), the module six cannot be found.  Do you know anyone with experience setting up Ceilometer in production for Juno?18:32
zaitcevimkarrer: This just makes no sense. And once you're deep enough to use rm(1), what's the point of using yum to begin with?18:33
*** mahatic has quit IRC18:33
zaitcevimkarrer: Ending with "package resolution and conflict issues" is bizzarre.18:35
imkarrerzaitcev: All of our other packages are installed via yum.  I figure it should be trivial to install the ceilometer middleware in the same manner the keystone middleware is installed.18:35
*** mahatic has joined #openstack-swift18:35
zaitcevimkarrer: well, yeah... What's the OS?18:36
imkarrerzaitcev: RHEL6.518:36
zaitcevimkarrer: okay, I have that. Let's see18:36
zaitcevgo go gadget ssh18:37
notmynametsg: thanks for the requirements stuff. I talked to ttx this morning and got it taken care of upstream. so yeah our next step is doing it on stable/kilo and master in swift18:37
zaitcev[root@rhev-a24c-01 ~]# cat /etc/redhat-release18:37
zaitcevRed Hat Enterprise Linux Server release 6.6 (Santiago)18:37
imkarrer[root@wdc01sft003ccz020 ~]# cat /etc/redhat-release18:38
imkarrerRed Hat Enterprise Linux Server release 6.5 (Santiago)18:38
notmynameclayg: looks like you updated priority reveiws page18:38
notmynameclayg: the DO is the one we need in RC2 and the DO NOT is the one we don't?18:38
zaitcevso, running "yum install openstack-ceilometer-common openstack-ceilometer-api python-ceilometerclient"18:39
*** mmcardle has joined #openstack-swift18:39
zaitcevokay, so installation had no issues... The middleware is actually in the shared code18:44
zaitcev[root@rhev-a24c-01 ~]# rpm -qf /usr/lib/python2.6/site-packages/ceilometer/objectstore/swift_middleware.py18:44
zaitcevpython-ceilometer-2014.1.2-1.el6.noarch18:44
zaitcevnow to configure18:44
*** MVenesio has joined #openstack-swift18:45
imkarrerSo the middleware comes from python-ceilometer?  Okay, thats probably much more lightweight than ceilometer-common, less packages to confict with...18:47
zaitcevYou're thinking about it all wrong. There should be no conflicts period18:49
zaitcevDon't even go there18:49
zaitcevThe python-ceilometer is installed automatically as a dependency. And if you use the corrent repositories for the basic distro, there will be no conflicts of any kind. It's all tested together by the QA.18:50
zaitcevAnyway, since I never installed Ceilometer before, I have no idea what I'm doing here18:51
notmynamezackmdavis wrote an awesome blog post on how erasure codes work: https://swiftstack.com/blog/2015/04/20/the-foundations-of-erasure-codes/18:51
imkarrerzaitcev: python-ceilometer gets installed as a dependency of swift?18:51
zaitcevimkarrer: no, openstack-ceilometer pulls it18:52
imkarrerzaitcev: I am more confused now.  I do not have access in my repo list openstack-ceilometer package, only openstack-ceilometer-common.18:53
*** pberis has joined #openstack-swift18:55
zaitcevimkarrer: sorry, I was imprecise. The openstack-ceilometer is built as a bunch of subpackages, which include -common, -api, etc. I presume you'll need the -api one for sure, because it contains the REST server.18:55
zaitcevAh-ha, here's the first problem - oslo.config.cfg.ConfigFilesNotFoundError: Failed to read some config files: /usr/share/ceilometer/ceilometer-dist.conf,/etc/ceilometer/ceilometer.conf18:57
imkarrerzaitcev: I already have ceilometer api running on a separate node.  I am trying to configure the ceilometer middleware so that I can see the three notifications which swift sends back to ceilometer.  The other ceilometer statistics (pollsters) are trivial.18:58
zaitcevimkarrer: right, so I guess only openstack-ceilometer-common is needed and then some kind of agent or whatnot.18:59
zaitcevimkarrer: I added swift to ceilometer group for now.  ceilometer:x:166:ceilometer,swift18:59
zaitcevimkarrer: also had to do chgrp ceilometer /var/log/ceilometer && chmod 775 /var/log/ceilometer19:01
imkarrerzaitcev: so what yum repos have you install thus far? using the openstack-ceilometer-common led to package confict issues for me.19:02
zaitcevoh, great - Apr 20 15:02:27 rhev-a24c-01 ceilometer.openstack.common.rpc.common: AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 9 seconds.19:02
imkarrerSo you are trying to hit the rabbitmq :D19:03
imkarrerI was able to get that far in my dev environment, but not with the yum packages19:03
zaitcevimkarrer: http://paste.openstack.org/show/204911/19:04
*** mmcardle has quit IRC19:04
*** silor1 has joined #openstack-swift19:05
zaitcevimkarrer: no, wait, that's useless. all the fake repo names19:05
imkarrerzaitcev: I mean packages19:06
*** silor has quit IRC19:06
*** pberis has quit IRC19:07
*** pberis has joined #openstack-swift19:08
zaitcevimkarrer: here is "yum repolist -v": http://paste.openstack.org/show/204912/   There's a lot of historic junk there, but really what you need is 1) RHEL 6, 2) EPEL, 3) RDO. Together they should never cause any conflicts.19:08
imkarrerzaitcev: but what package did you install to get the ceilometer middleware?19:10
zaitcevimkarrer: I just did " 1125  yum search ceilometer19:12
zaitcev 1126  yum install openstack-ceilometer-common openstack-ceilometer-api python-ceilometerclient"19:12
zaitcevimkarrer: result is like http://paste.openstack.org/show/204913/19:13
imkarrerThank you for all of your help today zaitcev,  I really appreciate another set of eye on this.19:13
imkarrereyes* :D19:13
zaitcevThe eagle-eyed among us might notice that I have openstack-swift-2.2.2-1.el6.noarch, which is not officially available (yet). It's a backport that I built from a Fedora RPM. However, I'm pretty sure that ceilometer middleware ought to work with the official Swift 1.13.1.19:14
*** tsg has quit IRC19:14
zaitcevHonestly, I have no clue what I'm doing with Ceilometer. But all OpenStack work about the same way... There's a REST front-end, a scheduler or some deamonistic thing, maybe a service daemon or two, and a bunch of agents.19:16
*** krtaylor has left #openstack-swift19:16
zaitcevThe RPMs for them typically are built from one SRPM. That thing has a bunch of shared code, that ends in something like python-manila. Every derived subpackage depends on that and only contains things like sample configs, docs, and maybe binaries.19:17
zaitcevSwift is (of course, haha) different in that there's no python-swift RPM. We keep all our common stuff in openstack-swift.19:18
*** pberis has quit IRC19:21
*** pberis has joined #openstack-swift19:22
*** stevemar has joined #openstack-swift19:22
stevemarwhats up with notifications and swift? seems like it doesn't use oslo messaging19:23
*** pberis has quit IRC19:25
*** pberis has joined #openstack-swift19:26
*** Fin1te has joined #openstack-swift19:26
zaitcevnotifications about what?19:27
*** minwoob has quit IRC19:29
*** aix has quit IRC19:29
*** imkarrer__ has joined #openstack-swift19:31
stevemarzaitcev, event notifications, and use auth_token and audit middleware19:32
*** wbhuber has quit IRC19:33
stevemarbut i guess cause it's still used as a standalone, so it's not using oslo messaging and other keystone bits19:33
*** imkarrer has quit IRC19:34
cschwedestevemar: there are no event notifications in swift?19:35
*** annegentle has quit IRC19:36
*** bill_az has joined #openstack-swift19:36
stevemarcschwede, well you can add keystonemiddleware's audit code to nova/cinder/etc, and it'll send off a notifications using oslo messaging19:38
stevemari think for all api requests that are in the mapping configuration (in pycadf)19:39
*** pberis has quit IRC19:43
*** pberis has joined #openstack-swift19:44
*** pberis has quit IRC19:45
*** pberis has joined #openstack-swift19:46
*** lpabon has quit IRC19:48
*** pberis has quit IRC19:50
claygnotmyname: I was trying to highlight there's some contraversary between patch 174875 and patch 17507619:56
patchbotclayg: https://review.openstack.org/#/c/174875/19:56
*** dencaval has quit IRC19:56
claygnotmyname: we don't either for rc2 - or said another way - we need some combination of both19:56
*** tsg has joined #openstack-swift19:56
claygnotmyname: I'm going to try and pull out the non-contraversial fixes in patch 175076 and resubmit19:57
patchbotclayg: https://review.openstack.org/#/c/175076/19:57
claygnotmyname: depending on how that goes I may try and combine some sane combination of patch 174875 with what's left19:57
patchbotclayg: https://review.openstack.org/#/c/174875/19:57
claygnotmyname: if I was trying to manage all these balls up in the air I'd focus on pyeclib 1.0.7 and the reconstruct patch - and leave the others alone - i'll catch you up with the issues acoles identified with my revert fix next time we're high bandwidth19:58
*** silor1 has quit IRC20:08
*** tsg has quit IRC20:12
*** pberis has joined #openstack-swift20:19
*** tsg has joined #openstack-swift20:28
*** imkarrer_ has joined #openstack-swift20:34
*** tsg has quit IRC20:35
*** imkarrer__ has quit IRC20:38
openstackgerritSamuel Merritt proposed openstack/swift: Make RingBuilders deep-copy-able  https://review.openstack.org/17556720:39
openstackgerritClay Gerrard proposed openstack/swift: Exclude local_dev from sync partners on failure  https://review.openstack.org/17507620:47
*** proteusguy has quit IRC20:52
*** annegent_ has joined #openstack-swift20:55
*** stevemar has quit IRC20:58
*** mahatic has quit IRC21:07
*** mmcardle has joined #openstack-swift21:29
*** mmcardle has quit IRC21:29
*** Fin1te has quit IRC21:33
*** Guest46134 has joined #openstack-swift21:35
*** Guest46134 is now known as _alejandrito_21:39
_alejandrito_hi everyone, is there a chance to make a container PUBLIC WRITABLE on swift JUNO / KILO without using tempURL ???21:41
*** erlon has quit IRC21:41
swifterdarrell_alejandrito_: it actually surprised a couple of us, but you can set a container write ACL to ".r:*" to allow public writes21:42
werhrm, I had to add that ability.  Last time I checked that wasn't a supported permission.  But I'm running an old version of swift now.21:42
claygswifterdarrell: so we have a doc bug?  "Also, .r designations aren’t allowed in headers whose names include the word ‘write’."21:43
swifterdarrell_alejandrito_: I mean, your auth middleware may or may not like it, but it's worked at least once with some auth middleware, YMMV, etc etc21:44
swifterdarrellclayg: yes?21:44
_alejandrito_swifterdarrell, clayg so ... i didnt found any documentation about doing a container writable21:44
claygContainer POST failed: http://saio:8080/v1/AUTH_test/test 400 Bad Request   Referrers not allowed in write ACL: '.r:*'21:44
swifterdarrellclayg: not 100% sure what swift version klrmn was using and/or whether tempauth or something that filters .r for write ACLs wasn't active...21:45
swifterdarrell_alejandrito_: sounds like it may not work to set a ".r:*" write ACL...but do try it and see21:46
_alejandrito_swifterdarrell, clayg wow ok, so i can try it in juno NOW, you say it might work ?21:46
swifterdarrellclayg: ya, that's disallowed in swift.common.middleware.acl.clean_acl()21:47
swifterdarrell_alejandrito_: sure, try it and it might work, but it looks like at least some code tries to make it NOT work, so... who knows?!21:48
swifterdarrell_alejandrito_: your own experimentation will trump what anyone says, in any case :)21:48
_alejandrito_swifterdarrell, the header would be X-Container-Write ?21:48
*** tsg has joined #openstack-swift21:49
*** MVenesio_ has joined #openstack-swift21:51
*** MVenesio has quit IRC21:54
*** kota_ has joined #openstack-swift21:54
kota_good morning21:55
_alejandrito_swifterdarrell, clayg Referrers not allowed in write ACL: '.r:*'21:59
swifterdarrell_alejandrito_: bummer :(21:59
swifterdarrell_alejandrito_: I checked and our auth middleware was changed to allow it to let swift3 middleware support a canned ACL of public-read-write on a "bucket"22:00
mattoliverauMorning22:02
_alejandrito_swifterdarrell, clayg so, it was an in-house mod you did ?22:02
_alejandrito_swifterdarrell, can you share it to me ? its exactly what i need22:02
_alejandrito_swifterdarrell, ;D22:02
swifterdarrell_alejandrito_: no, but the idea is to just modify swift.common.middleware.acl.check_acl() to not disallow it22:03
notmynamekota_: mattoliverau: hello22:03
kota_nomyname, mattoliverau: hi :)22:04
_alejandrito_swifterdarrell, swift.common.middleware.acl.check_acl() doesnt exists on juno22:06
swifterdarrell_alejandrito_: is "def check_acl" somewhere else in the code tree?22:07
swifterdarrell_alejandrito_: maybe it moved or something22:07
_alejandrito_swifterdarrell, nope, it doesnt exists in junor nor kilo22:08
openstackgerritSamuel Merritt proposed openstack/swift-specs: Symlinks in Swift  https://review.openstack.org/17360922:09
swifterdarrell_alejandrito_: Perhaps search for some portion of the response string you get?22:10
swifterdarrell_alejandrito_: e.g.  "grep -r 'not allowed in write' ."22:10
swifterdarrell_alejandrito_: surely, somewhere, there is code that says "not allowed in write"22:10
notmynameswifterdarrell: _alejandrito_: clean_acl, not check_acl22:10
notmynameright?22:10
swifterdarrell_alejandrito_: ^^^^^^22:10
swifterdarrell_alejandrito_: ya, I misspoke?22:11
swifterdarrell_alejandrito_: grep is 100% better than me22:11
swifterdarrellnotmyname: thx!22:11
notmynamekota_: do you mind if I propose https://review.openstack.org/#/c/175299/ to stable/kilo for RC2 and manage that (instead of you doing it)?22:11
kota_notmyname: yes, right now, I'm looking at acoles question22:12
notmynamekota_: ok. yeah if there are oustanding questions, I'd like to see them resolved first22:13
kota_notmyname: clayg already pushed the unittest but the unittest passed with v1.0.6 :P22:13
_alejandrito_notmyname, so if i want for clean_acl to allow me to set PUBLIC WRITE ACL show i remove this ones ? https://github.com/openstack/swift/blob/stable/juno/swift/common/middleware/acl.py#L101-L10322:13
notmyname_alejandrito_: yeah, looks like it22:13
kota_notmyname: and I found the change on the unittest we need (maybe) and I'll push it ASAP22:13
notmynamekota_: ok, thanks22:13
_alejandrito_notmyname, swifterdarrell commenting out this 3 lines https://github.com/openstack/swift/blob/stable/juno/swift/common/middleware/acl.py#L101-L103 worked like a charm22:16
swifterdarrell_alejandrito_: sweet!22:16
*** imkarrer_ has quit IRC22:20
notmynameRC2 patch list updated on https://wiki.openstack.org/wiki/Swift/PriorityReviews22:21
notmynamehttps://review.openstack.org/#/c/175299/ needs to land on master, then be proposed to stable/kilo22:21
egonAnyone here looked at the EMC Vipr thing?22:22
notmynamethen the 4 patches listed (the 3 on the wiki page plus the one that needs to be proposed) can land on stable/kilo22:22
notmynameI'll need to update the commit messages for them to add a LP bug reference (Which I'll also need to create for each of them)22:22
notmynamesummary is, review https://review.openstack.org/#/c/175299/ and land it on master, and then i'll take care of the button clicking for backports22:23
notmynamealso, please ping me ASAP if there is something else that needs to be in RC2 for the release22:23
notmynameegon: if you've got access to one, I'd love to see what happens when you run the swift functional tests against it22:24
*** tsg has quit IRC22:25
kota_oops, my assumption seemed wrong, now confuesed as same as acoles. now thinking...22:26
openstackgerritMinwoo Bae proposed openstack/swift: Included step in development_saio.rst for installing dependencies in requirements.txt.  https://review.openstack.org/17559422:28
dfgq22:31
dfgoops22:31
kota_notmyname: ok, I confirm clayg's patch is good (the test pass was my mistake to install liberasurecode) so I think the newest patch set is ready to land on path 17529922:38
kota_s/path/patch/22:38
*** jrichli has quit IRC22:41
*** _alejandrito_ has quit IRC22:46
*** pberis has quit IRC22:47
*** km has joined #openstack-swift22:50
kota_I'm leaving a while for breakfast.22:52
*** kota_ has quit IRC22:52
notmynamekota_: I'm about to look at that. thanks for pushing another copy22:52
notmyname* patch set22:52
*** annegent_ has quit IRC22:57
notmynameclayg: did you write the tests on https://review.openstack.org/#/c/175299 ?22:57
notmynamehmm...so my turn now23:08
notmynamewhat's the right way to upgrade liberasurecode once you've had it installed via pyeclib before?23:09
*** lcurtis has quit IRC23:22
*** esker has quit IRC23:27
*** chlong has joined #openstack-swift23:34
claygnotmyname: uninstall pyeclib and delete /usr/local/lib/liberasure.*23:35
claygthen reinstall pyeclib23:35
claygnotmyname: ^ WOMM23:35
claygnotmyname: yeah i wrote the test - what of it!23:35
notmynameyeah, that's what I did too. just wondered if there was a better day23:35
*** kei_yama has joined #openstack-swift23:35
notmynameclayg: you've got some magic numbers in there. wondering what they are. also, maybe should have a co-authored-by line23:36
claygnotmyname: i don't want anyone to pin that cappy code on me - much better in my mind if the blame kota!23:36
*** MVenesio_ has quit IRC23:36
notmynamelol23:36
claygyeah I think the test name sorta hints at the magic - reconstruct *parity* with *data* node down23:37
claygso the broken_body is a parity index (because the schema is 10+4) and the error_index is the down'd data node (randomint 0, ec_ndata-1)23:38
notmynameclayg: what about [:-454]23:40
claygoh i just try not to make it easy for fragment or segment size to align with the client data23:41
claygI started doing that in all my tests - it's just triming some random number of bytes23:41
claygthe test I copied was doing 777 I think23:41
notmynameoh23:41
notmynamemostly it was thet (the -454) that was weird to me. the pop(-4) makes sense now23:43
notmynamethanks23:43
claygnotmyname: might could all use comments - let me know if there's a good time to add them - or if kota can do it23:43
claygnotmyname: more than likely after it gets merged no one will look at that test again unless it fails :\23:44
*** david-ly_ has joined #openstack-swift23:44
claygnotmyname: that's how good tests work right?!23:44
*** jamielennox|away is now known as jamielennox23:45
*** tsg has joined #openstack-swift23:45
*** david-ly_ is now known as david-lyle23:46
openstackgerritSamuel Merritt proposed openstack/swift: SAIO instructions: ensure ~/bin exists before copying into it  https://review.openstack.org/17561323:47
openstackgerritJohn Dickinson proposed openstack/swift: Use reconstruct insetad of decode/encode  https://review.openstack.org/17529923:49
notmynameclayg: I added comments ^23:49
*** annegentle has joined #openstack-swift23:57
*** openstackgerrit has quit IRC23:58
*** openstackgerrit has joined #openstack-swift23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!