Monday, 2017-07-24

*** catintheroof has joined #openstack-swift00:33
*** kiennt has joined #openstack-swift00:35
*** ukaynar has quit IRC00:49
openstackgerritHangdong Zhang proposed openstack/python-swiftclient master: Update the documentation link for doc migration  https://review.openstack.org/48602901:07
*** frank_young has joined #openstack-swift01:14
*** vint_bra has quit IRC01:26
*** catintheroof has quit IRC02:22
*** tovin07_ has joined #openstack-swift02:33
*** frank_young has quit IRC03:06
*** mtreinish has quit IRC03:14
*** frank_young has joined #openstack-swift03:16
*** mtreinish has joined #openstack-swift03:20
*** frank_young has quit IRC03:21
*** frank_young has joined #openstack-swift03:27
*** gkadam has joined #openstack-swift03:30
*** links has joined #openstack-swift03:46
*** frank_young has quit IRC03:48
*** klrmn has quit IRC03:48
*** kei_yama has quit IRC03:52
*** ukaynar has joined #openstack-swift03:54
*** frank_young has joined #openstack-swift03:56
*** kei_yama has joined #openstack-swift04:02
*** frank_young has quit IRC04:15
*** kei_yama has quit IRC04:24
*** frank_young has joined #openstack-swift04:25
openstackgerritTovin Seven proposed openstack/swift master: OSprofiler in OpenStack Swift  https://review.openstack.org/46831604:27
*** frank_young has quit IRC04:30
*** ukaynar has quit IRC04:44
*** Dinesh_Bhor has joined #openstack-swift04:56
*** frank_young has joined #openstack-swift05:00
*** wasmum has quit IRC05:01
*** wasmum has joined #openstack-swift05:02
*** kei_yama has joined #openstack-swift05:04
*** tonanhngo has joined #openstack-swift05:10
*** bkopilov_ has joined #openstack-swift05:10
*** kei_yama has quit IRC05:16
*** frank_young has quit IRC05:18
*** kei_yama has joined #openstack-swift05:20
*** psachin has joined #openstack-swift05:30
*** kiennt has quit IRC05:32
*** cshastri has joined #openstack-swift06:00
*** frank_young has joined #openstack-swift06:02
*** kiennt has joined #openstack-swift06:04
*** zaitcev has quit IRC06:08
*** kong has quit IRC06:17
*** kiennt has quit IRC06:18
*** kiennt has joined #openstack-swift06:18
*** rcernin has joined #openstack-swift06:23
*** PavelK has joined #openstack-swift06:40
*** cschwede has joined #openstack-swift06:57
*** ChanServ sets mode: +v cschwede06:57
*** ChubYann has quit IRC07:00
*** tonanhngo has quit IRC07:13
*** wasmum has quit IRC07:28
*** wasmum has joined #openstack-swift07:28
PavelKHi, I would like to ask about patch 326903. We often update objects that has X-Delete-At header before they are expired and with this patch expirer tries one week to delete an old object. I suggest to HEAD an object when 412 Precondition Failed is returned and remove an object from expiration queue if its timestamp is newer than an entry has in expiration queue. Another ideas?07:35
patchbothttps://review.openstack.org/#/c/326903/ - swift - Raise 412 response on expirer (MERGED)07:35
*** skudlik has joined #openstack-swift07:46
*** geaaru has joined #openstack-swift07:47
*** tonanhngo has joined #openstack-swift08:04
*** tonanhngo has quit IRC08:08
*** wasmum has quit IRC08:13
*** wasmum has joined #openstack-swift08:14
*** oshritf has joined #openstack-swift08:14
*** oshritf has quit IRC08:19
*** tonanhngo has joined #openstack-swift08:32
*** tonanhngo has quit IRC08:37
*** oshritf has joined #openstack-swift08:59
*** jaosorior has joined #openstack-swift09:02
*** tonanhngo has joined #openstack-swift09:24
*** tonanhngo has quit IRC09:28
*** tovin07_ has quit IRC10:05
*** kong has joined #openstack-swift10:16
*** tonanhngo has joined #openstack-swift10:18
*** tonanhngo_ has joined #openstack-swift10:20
*** tonanhngo has quit IRC10:23
*** tonanhngo_ has quit IRC10:25
*** kiennt has quit IRC10:33
*** Venkata has joined #openstack-swift10:41
*** tonanhngo has joined #openstack-swift11:02
*** Dinesh_Bhor has quit IRC11:07
*** tonanhngo has quit IRC11:07
*** Venkata has quit IRC11:08
*** kei_yama has quit IRC11:11
openstackgerritPavel Kvasnička proposed openstack/swift master: Objects are deleted from additional handoffs  https://review.openstack.org/47044011:33
*** tonanhngo has joined #openstack-swift11:34
*** oshritf has quit IRC11:35
*** tonanhngo has quit IRC11:39
*** oshritf has joined #openstack-swift11:52
*** vint_bra has joined #openstack-swift12:30
xrb_hi all,12:32
xrb_I am doing a few tests with async processing in a middleware (using eventlet.posthooks), which is working fine so far, except the following issue12:33
xrb_In the posthook I need to grab the request data (env['wsgi.input']) and make it seekable, which I can do by calling req.make_body_seekable()..12:34
xrb_Although here I am getting troubles. Be it in __call__ or in the posthook, I get errors when calling this method.12:35
xrb_E.g. currently getting a client disconnect error ('The client disconnected while sending the body'). But I am pretty sure it is a server problem, because it comes through when I remove this call...12:36
xrb_Any hint appreciated...12:37
*** gkadam has quit IRC12:41
*** catintheroof has joined #openstack-swift12:42
openstackgerritPavel Kvasnička proposed openstack/swift master: IDEA: Handle 412 response on expirer when object updated  https://review.openstack.org/48659912:47
*** chlong has joined #openstack-swift12:53
*** tonanhngo has joined #openstack-swift13:00
*** vint_bra has quit IRC13:01
*** lucasxu has joined #openstack-swift13:11
*** links has quit IRC13:20
xrb_Apparently the problem is specific to calling req.make_body_seekable() from the posthook. Maybe the body stream is not available from the hook callback. Hence currently I must call the hook callback synchronously from the __call__ method...13:34
xrb_I would like to find a solution to do it asynchronously though (currently doing thumbnail generation from the posthook, I don't want this to slow down my main request).13:35
*** Dinesh_Bhor has joined #openstack-swift13:40
*** MVenesio has joined #openstack-swift13:53
*** vint_bra has joined #openstack-swift13:59
*** abhitechie has joined #openstack-swift14:14
openstackgerritGábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/common  https://review.openstack.org/34278114:15
openstackgerritGábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/container  https://review.openstack.org/34280814:15
openstackgerritGábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/obj tests  https://review.openstack.org/34283014:15
openstackgerritGábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/common/middleware  https://review.openstack.org/34277014:15
openstackgerritGábor Antal proposed openstack/swift master: Update hacking and enable off-by-default checks  https://review.openstack.org/48662714:15
*** abhitechie has quit IRC14:21
*** frank_yo_ has joined #openstack-swift14:35
*** frank_young has quit IRC14:36
*** cshastri has quit IRC14:39
*** frank_yo_ has quit IRC14:50
*** zaitcev has joined #openstack-swift14:55
*** ChanServ sets mode: +v zaitcev14:55
*** sanchitmalhotra has quit IRC14:57
*** sanchitmalhotra has joined #openstack-swift15:01
*** lucasxu has quit IRC15:03
*** MVenesio has quit IRC15:04
*** MVenesio has joined #openstack-swift15:04
*** skudlik has quit IRC15:05
*** klrmn has joined #openstack-swift15:10
*** geaaru has quit IRC15:13
*** chlong_ has joined #openstack-swift15:21
*** chlong has quit IRC15:24
tdasilvatimburke: still investigating failure on probe tests after patch 486264 merged: http://paste.openstack.org/show/616316/   :\15:29
patchbothttps://review.openstack.org/#/c/486264/ - swift - Close stdout *after* binding ports (MERGED)15:29
*** rcernin has quit IRC15:34
*** wasmum has quit IRC15:36
*** wasmum has joined #openstack-swift15:39
*** kong has quit IRC15:45
notmynamegood morning15:55
*** rcernin has joined #openstack-swift15:58
notmynamemattoliverau: thanks for the bug triage work16:01
tdasilva+116:02
*** JimCheung has joined #openstack-swift16:12
*** chlong__ has joined #openstack-swift16:12
*** chlong_ has quit IRC16:13
*** klrmn has quit IRC16:15
*** SkyRocknRoll has joined #openstack-swift16:16
*** frank_young has joined #openstack-swift16:24
timburketdasilva: seems like https://github.com/openstack/swift/blob/master/test/probe/common.py#L361 is the culprit; i wonder why we aren't waiting...16:30
timburkewas added in https://github.com/openstack/swift/commit/0efac0c ...16:31
*** chlong_ has joined #openstack-swift16:39
*** chlong__ has quit IRC16:41
openstackgerritTim Burke proposed openstack/python-swiftclient master: Update the documentation link for doc migration  https://review.openstack.org/48602916:50
*** chlong_ has quit IRC17:08
*** chlong_ has joined #openstack-swift17:09
*** oshritf has quit IRC17:13
*** vint_bra has quit IRC17:16
*** klrmn has joined #openstack-swift17:16
*** ChubYann has joined #openstack-swift17:18
*** vint_bra has joined #openstack-swift17:18
*** frank_young has quit IRC17:20
*** skudlik has joined #openstack-swift17:21
*** xrb_ has quit IRC17:23
*** xrb has joined #openstack-swift17:23
*** xrb has quit IRC17:23
*** skudlik has left #openstack-swift17:23
*** xrb has joined #openstack-swift17:24
*** xrb has quit IRC17:28
*** frank_young has joined #openstack-swift17:30
*** frank_young has quit IRC17:35
*** bkopilov has quit IRC17:44
*** bkopilov_ has quit IRC17:46
*** bkopilov_ has joined #openstack-swift17:49
*** bkopilov has joined #openstack-swift17:50
*** vint_bra has quit IRC17:51
*** mvk has quit IRC17:52
*** vint_bra has joined #openstack-swift17:55
*** gkadam has joined #openstack-swift18:00
tdasilvatimburke: what's confusing is that, IIRC, very few calls to Manager.start() have a wait=True on them18:01
timburketdasilva: it defaults to true: https://github.com/openstack/swift/blob/master/swift/common/manager.py#L24318:02
tdasilvaheh, that's probably why18:04
timburkei actually started by running around slapping `wait=True` on everything, then got confused about why it wasn't having the desired affect :-)18:05
tdasilvahehe18:06
tdasilvayeah, that was my first attempt to18:06
tdasilvabtw..running tests now with the change to common.py18:07
timburkepart of why i figured the comment was the most valuable part of patch 48626418:07
patchbothttps://review.openstack.org/#/c/486264/ - swift - Close stdout *after* binding ports (MERGED)18:07
*** wasmum has quit IRC18:09
*** mathiasb has quit IRC18:09
*** wasmum has joined #openstack-swift18:10
*** mathiasb has joined #openstack-swift18:16
*** psachin has quit IRC18:16
*** rcernin has quit IRC18:16
*** SkyRocknRoll has quit IRC18:19
openstackgerritJohn Dickinson proposed openstack/swift master: 2.15.0 authors and changelog update  https://review.openstack.org/48583918:23
notmynamemy current thoughts on tagging releases is tagging swiftclient tomorrow night or wednesday morning (timur is working on https://review.openstack.org/#/c/475038/ now)18:25
patchbotpatch 475038 - python-swiftclient - Allow for uploads from standard input.18:25
notmyname... and tagging swift towards the end of the week18:25
openstackgerritMerged openstack/swift master: Stop using deprecated 'message' attribute in Exception  https://review.openstack.org/48330018:32
notmynamepeople using swift: "Just created a @coreos #rkt image repository on Openstack Swift. Very simple and effective. #kiss #ilike" (https://twitter.com/vincentvdk/status/889552591650254849)18:33
notmynametdasilva: now how do we get luis to do that? ;-)18:34
tdasilvaaren't container repos perfect for object storage?18:35
notmynameyes!18:35
*** gkadam has quit IRC18:38
*** wasmum has quit IRC18:41
*** wasmum has joined #openstack-swift18:43
*** itlinux has joined #openstack-swift18:53
openstackgerritJohn Dickinson proposed openstack/swift master: 2.15.0 authors and changelog update  https://review.openstack.org/48583919:04
openstackgerritTimur Alperovich proposed openstack/python-swiftclient master: Turn stdin uploads into SLO above 10MB.  https://review.openstack.org/47861119:13
openstackgerritTimur Alperovich proposed openstack/python-swiftclient master: Allow for uploads from standard input.  https://review.openstack.org/47503819:13
*** frank_young has joined #openstack-swift19:13
openstackgerritMathias Bjoerkqvist proposed openstack/swift master: Retrieve encryption root secret from Barbican  https://review.openstack.org/36487819:14
*** frank_young has quit IRC19:17
timburketimur: may as well rebase now; we can always look at the diff between patchsets 3 and 4 later if we really want19:41
openstackgerritMerged openstack/python-swiftclient master: Update the documentation link for doc migration  https://review.openstack.org/48602919:53
*** JimCheung has quit IRC19:54
*** xrb has joined #openstack-swift20:07
*** JimCheung has joined #openstack-swift20:10
notmynamewhoa. if tdasilva like the current revision of https://review.openstack.org/#/c/364878/, that could go in this release too!20:13
patchbotpatch 364878 - swift - Retrieve encryption root secret from Barbican20:13
*** JimCheung has quit IRC20:14
tdasilvanotmyname: I'm going to check it out, it would be great if acoles, jrichli or whoever has worked with encryption before could take a look20:16
notmynameacoles is out today. will be back tomorrow20:16
notmynamemeaning ~12 hours from now20:17
tdasilvamathiasb: one issue I ran into is that when trying to setup the functional tests, I configured keystone with the users necessary in /etc/swift/test.conf, then i had 8 tests in versioning fail. I'm certain it has nothing to do with your work, but I couldn't figure out what I was doing wrong, would be cool to know if you were able to run tests without those errors, maybe we can compare notes20:19
*** JimCheung has joined #openstack-swift20:26
timurtimburke: ok, I'll work on that20:30
*** JimCheung has quit IRC20:30
xrbany hint about a workaround regarding env['wsgi.input'] which cannot be made seekable from a eventlet.posthooks callback?20:31
xrb(produces a client disconnect, maybe because the request handling is already finished, hence the input stream might be in a different state)20:31
xrbmaybe there is still a workaround...20:32
mathiasbtdasilva: do you have both tempauth and keystoneauth in the pipeline?20:33
*** JimCheung has joined #openstack-swift20:35
openstackgerritThiago da Silva proposed openstack/swift master: ready yet? nope, please wait!  https://review.openstack.org/48674520:35
tdasilvamathiasb: checking, didn't think so, but checking20:36
tdasilvamathiasb: btw, made a small change to your script: https://gist.github.com/thiagodasilva/51a7f627a53e4721d8ae662a53e8e40520:37
openstackgerritOpenStack Proposal Bot proposed openstack/python-swiftclient master: Updated from global requirements  https://review.openstack.org/8925020:37
tdasilvabasically just make sure to export the env SAIO_BLOCK_DEVICE20:37
tdasilvawas also planning on adding additional keystone users to pre-configure the test.conf file20:38
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements  https://review.openstack.org/8873620:38
tdasilvamathiasb: no tempauth20:38
notmynamexrb: IIRC you've got an async process you want to perform on the object content, so you want to rewind the stream?20:38
mathiasbtdasilva: the script needs some changes to pull the latest patch set (26 vs 29), and also to install the dependencies using [extras]20:39
tdasilvamathiasb: cool, will do that20:40
tdasilvamathiasb, timur: so how does one make use of [extras] to install the dependencies20:40
tdasilva?20:40
tdasilvasudo pip install swift[kms-encryption] ?20:40
tdasilvakms_keymaster20:41
mathiasbyes, that's what I used and documented20:42
tdasilvacool, thanks20:44
mathiasbtdasilva notmyname : this patch https://review.openstack.org/88736 proposes newer versions of e.g., cryptography to requirements.txt, but they are not being merged, any particular reason for that?20:49
patchbotpatch 88736 - swift - Updated from global requirements20:50
notmynamemathiasb: several :-)20:50
mathiasbcastellan seems to require cryptography>=1.6, but I put in the [extras] in the barbican patch so as not to mess with the main requirements20:50
mathiasboh :)20:51
notmynamethe shortest answer is that we aren't going to raise our dependencies when they don't conflict (ie don't force operators to repackage a lot of stuff when the old is available and doesn't conflict). however, if we need to add something or updates something, it must match what's in global requirements20:51
notmynameI'll admit that we don't do a great job about continually validating the intersection of our existing requirements against global requirements. turns out that the gate ignores our requirements file anyway and just uses global requirements20:52
notmynamebut the reasons for sticking with ">=some.older.version" instead of whatever is latest is because I don't want to force repackaging work on deployers when some other project starts needing a different version, especially if the older version is what's distro-provided20:53
timburkenot even that; it uses upper-constraints as i recall20:53
notmynameright20:53
notmynamemathiasb: so for cryptography, if you're bumping the requirement, just make it an exact string match for what's in global requirments and dont' mess with the other lines in the [test-]requirments.txt files20:54
mathiasbok, so bumping the cryptography version "shouldn't" be a big problem since it's been tested in the gate?20:54
mathiasbok20:55
notmynamecorrect. the gate is already testing what's in upper constraints, so if the gate is currently passing, then updating the cryptography line in requirements.txt won't change gate passingness20:55
* notmyname is not sure if "gate passingness" is a real word20:56
* notmyname hopes it translates!20:56
xrbnotmyname: yes, at some point during processing, the lib I am using for thumbnail generation does a seek on the input..20:57
xrbnotmyname: that's the reason why I need the input to be seekable. But it doesn't work like in the __call__ method...20:58
notmynamexrb: ok. so if the input stream is a few GB, then that would all have to be buffered to make it seekable, right? wouldn't that severely limit the number of current connections a proxy server can handle? the proxy server intentionally doesn't do that right now20:58
notmynamexrb: so I don't know exactly why you're getting the error you're seeing, but I'm not surprised it doesn't work (for the above reason)20:59
notmynamexrb: what if you passed the object name (or full path) to your async process? would you be able to take that and reread the object for the image processing you want to do? eg feed the object name to a work queue somewhere instead of feeding the data stream directly. then each image worker grabs an object and does whatever21:00
xrbnotmyname: yes, it would be a nice solution.. I could just re-read the object, generate the thumbnail and create a new object...21:02
notmynamexrb: yeah. would also allow for some interesting separation of concerns (not running workers on the proxy servers, independently scaling workers, etc)21:02
xrbso the eventlet.posthooks would probably not be the right mechanism, would it? any standard queue mechanism integrated with the proxy? Or maybe a separate process like OpenWhisk triggered from the posthook?21:05
notmynameI think all of those would work :-)21:05
notmynamethe second is probably the simplest21:05
notmynamethe first (your current posthooks solution) has the advantage of only dumping something in the work queue after the client request is done21:06
notmynamebut if you triggered off a [201,202] response code being sent to the client, you'd be find21:06
*** MVenesio has quit IRC21:19
mathiasbtdasilva: I ran the functests and I did get 23 skipped but 0 failed21:26
openstackgerritTim Burke proposed openstack/swift master: Add multiple worker processes strategy to reconstructor  https://review.openstack.org/47841621:26
*** mvk has joined #openstack-swift21:28
mathiasbtdasilva: I didn't create the users manually, but just removed /etc/swift/test.conf and ran .functests and let it create the keystone users automatically21:35
torgomaticthings to consider with queue solutions: (a) external request finishes, but queue service is momentarily down; what happens to your notification?21:50
torgomatic(b) queue service is down for a long time. does your system fill up and fall over?21:50
torgomatic(c) client request finishes, proxy dies at that instant; do you get a notification out?21:51
torgomatic(d) client request finishes, proxy gets notification to external queue, but reply from queue -> proxy is lost; do you retry?21:52
torgomaticthere's probably a bunch more junk to consider too21:52
notmynametimur says that xrb could use a container db walker to find info without worrying about queue options21:54
notmynametimur: ...if only there were one out there that could be used...21:55
*** kong has joined #openstack-swift21:55
timurI think notmyname is referring to this library that SwiftStack wrote and uses to index swift objects in elasticsearch: https://github.com/swiftstack/container-crawler21:57
timurxrb: depending on the image format, it may also be possible to tweak it to avoid seeks. I did something like that to extract EXIF metadata out of the jpegs: https://github.com/timuralp/exif-stream21:58
*** itlinux has quit IRC22:04
notmynametimur: thanks :-)22:16
*** frank_young has joined #openstack-swift22:19
*** frank_young has quit IRC22:24
*** vint_bra has quit IRC22:44
tdasilvamathiasb: was not aware that .functests would create keystone users for me automagically23:01
tdasilvaneat!23:01
*** xrb_ has joined #openstack-swift23:02
*** xrb has quit IRC23:02
*** ukaynar has joined #openstack-swift23:05
*** itlinux has joined #openstack-swift23:05
*** itlinux has quit IRC23:10
openstackgerritTimur Alperovich proposed openstack/python-swiftclient master: Allow for uploads from standard input.  https://review.openstack.org/47503823:19
openstackgerritTimur Alperovich proposed openstack/python-swiftclient master: Turn stdin uploads into SLO above 10MB.  https://review.openstack.org/47861123:19
*** catintheroof has quit IRC23:21
timurtimburke: fixed up the rebase to include the documentation commit from notmyname. Sadly, I don't think doing functional tests makes sense for the first patch, as there are no functional tests of the shell.py functions23:29
timurI could try to figure out how to add something, but I think we could also do that in a follow-on change23:29
*** chlong__ has joined #openstack-swift23:36
*** chlong_ has quit IRC23:37
*** kei_yama has joined #openstack-swift23:41

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!