Thursday, 2021-01-14

openstackgerritTim Burke proposed openstack/swift master: s3api: Get rid of slo_enabled flag  https://review.opendev.org/c/openstack/swift/+/77068500:51
zaitcevDoes anyone else have a weird interaction of s3api and ratelimit?01:02
zaitcevOh, actually etc/proxy-server.conf-sample talks about it.01:04
openstackgerritTim Burke proposed openstack/swift master: Replace S3Request.location with S3Request.s3api_region  https://review.opendev.org/c/openstack/swift/+/77069001:20
timburke_i wonder if i should just take the region-for-the-sake-of-auth from the Authorization header, and stop looking at the configured location in S3Request entirely...01:22
openstackgerritTim Burke proposed openstack/swift master: Replace S3Request.location with S3Request.s3api_region  https://review.opendev.org/c/openstack/swift/+/77069001:23
timburke_(forgot about fixing up tests)01:24
*** hoonetorg has joined #openstack-swift01:42
timburke_zaitcev, note that prior to https://github.com/openstack/swift/commit/1db11df you kinda had to have ratelimit right of auth (or put up with swift requests double-counting against your limits, if you did a double-placement anyway)01:42
*** timburke_ has quit IRC01:43
*** baojg has quit IRC02:06
*** baojg has joined #openstack-swift02:06
openstackgerritTim Burke proposed openstack/swift master: prefactor: Pass s3api Config to S3Requests  https://review.opendev.org/c/openstack/swift/+/72909102:08
openstackgerritTim Burke proposed openstack/swift master: s3api: Get rid of slo_enabled flag  https://review.opendev.org/c/openstack/swift/+/77068502:08
openstackgerritTim Burke proposed openstack/swift master: Replace S3Request.location with S3Request.s3api_region  https://review.opendev.org/c/openstack/swift/+/77069002:08
_david_sohonetIf I'm not using sharding, is it correct to ignore the 'states' URL request param ? I'm seeing in proxy-server log lines ? I've not noticed it before. I assume it's added after the request hits the proxy-server ?02:53
*** rcernin has quit IRC03:04
*** rcernin has joined #openstack-swift03:25
*** rcernin has quit IRC03:27
*** rcernin has joined #openstack-swift03:31
*** rcernin has quit IRC03:36
*** rcernin has joined #openstack-swift03:37
*** rcernin has quit IRC03:38
*** rcernin has joined #openstack-swift03:38
*** rcernin has quit IRC03:39
*** rcernin has joined #openstack-swift03:39
*** psachin has joined #openstack-swift03:58
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-swift05:35
*** m75abrams has joined #openstack-swift06:06
*** gyee has quit IRC06:31
openstackgerritMatthew Oliver proposed openstack/swift master: Reclaim shards when root container is reclaimed  https://review.opendev.org/c/openstack/swift/+/77052906:41
*** dsariel has joined #openstack-swift06:54
*** rcernin has quit IRC07:40
*** rpittau|afk is now known as rpittau07:47
*** rcernin has joined #openstack-swift07:56
*** rcernin has quit IRC08:01
*** rcernin has joined #openstack-swift08:26
*** m75abrams has quit IRC09:45
*** m75abrams has joined #openstack-swift09:49
*** m75abrams has quit IRC09:51
*** m75abrams has joined #openstack-swift09:53
*** rcernin has quit IRC10:14
*** rcernin has joined #openstack-swift10:23
*** rcernin has quit IRC10:40
*** baojg has quit IRC11:09
*** baojg has joined #openstack-swift11:11
*** baojg has quit IRC11:11
*** baojg has joined #openstack-swift11:12
*** baojg has quit IRC11:12
*** baojg has joined #openstack-swift11:12
*** baojg has quit IRC11:13
*** baojg has joined #openstack-swift11:13
*** baojg has quit IRC11:14
*** baojg has joined #openstack-swift11:14
*** baojg has quit IRC11:14
*** baojg has joined #openstack-swift11:15
*** baojg has quit IRC11:15
openstackgerritAlistair Coles proposed openstack/swift master: s3api: Make allowable clock skew configurable  https://review.opendev.org/c/openstack/swift/+/77000511:53
*** rcernin has joined #openstack-swift11:55
openstackgerritAlistair Coles proposed openstack/swift master: Make test_verify_server independent of test env  https://review.opendev.org/c/openstack/swift/+/77079611:58
*** rcernin has quit IRC11:58
*** psachin has quit IRC12:39
*** takamatsu has joined #openstack-swift14:35
*** m75abrams has quit IRC14:48
acoles_david_sohonet:14:52
acoles_david_sohonet: yes, the 'states' param has no effect with unsharded containers, and yes it is added by the proxy to GET requests to backend container servers14:53
*** m75abrams has joined #openstack-swift15:07
*** biyiklioglu has joined #openstack-swift15:07
*** biyiklioglu has quit IRC15:16
*** biyiklioglu has joined #openstack-swift15:25
*** hoonetorg has quit IRC15:35
*** hoonetorg has joined #openstack-swift15:40
*** hoonetorg has quit IRC15:43
*** gyee has joined #openstack-swift16:22
*** dosaboy has quit IRC16:24
*** dosaboy has joined #openstack-swift16:25
*** hoonetorg has joined #openstack-swift16:45
*** m75abrams has quit IRC16:57
*** diablo_rojo has joined #openstack-swift17:04
*** rpittau is now known as rpittau|afk17:42
openstackgerritAlistair Coles proposed openstack/swift master: ssync: sync non-durable fragments from handoffs  https://review.opendev.org/c/openstack/swift/+/77004718:08
acoleswow, that ^^^ has conflicts with more than 50 other patches, could be time for some abandoning18:12
*** dosaboy has quit IRC18:21
*** diablo_rojo has quit IRC18:34
*** diablo_rojo has joined #openstack-swift18:43
*** timburke has joined #openstack-swift18:48
*** ChanServ sets mode: +v timburke18:48
timburkehmm. do we not have any tests that exercise somethig like `swift-init container.1 start`? i think that may have stopped working following https://review.opendev.org/c/openstack/swift/+/764137 ...18:51
*** dosaboy has joined #openstack-swift19:11
*** timburke_ has joined #openstack-swift19:12
*** ChanServ sets mode: +v timburke_19:12
claygi don't think i've ever used that interface - but i feel like I used to be able to say something like `swift-init status proxy.noauth`19:12
clayg`swift-init container status -c 1` works fine - maybe we could expand -c to accept strings (i.e. `swift-init status proxy -c noauth`)19:13
timburke_i think i only noticed the .N thing after you pointed out that `swift-init status proxy.noauth` is possible ;-)19:14
claygyeah, love that feature19:14
*** timburke has quit IRC19:15
claygbut `-c noauth` would be entirely acceptable - i always thought `container.1` was silly19:15
*** fingo has quit IRC20:15
timburke_huh. so proxy.noauth gets parsed in Server.__init__ (https://github.com/openstack/swift/blob/2.26.0/swift/common/manager.py#L445-L448) and gets used to limit our search in _find_conf_files (https://github.com/openstack/swift/blob/2.26.0/swift/common/manager.py#L505-L511)20:16
timburke_while -c/--config-num comes gets used as number around https://github.com/openstack/swift/blob/2.26.0/swift/common/manager.py#L541-L548 and really *does* have to be a int...20:16
*** fingo has joined #openstack-swift20:18
claygnah, just change the type of `-c` to be a string, set it to .conf and let _find_conf_files do it's thing (the index trick is lame)20:19
timburke_clayg, fwiw, you could just do `swift-init status proxy -c 1` *today* to have it only look at your noauth proxy20:20
claygyeah, but that's lame20:22
*** dosaboy has quit IRC21:00
*** dosaboy has joined #openstack-swift21:02
timburke_ugh. yeah, i need to get overall retries in the gate for func tests -- it sucks to have a patch wait in the check queue for 25hrs just to fail because of container-listing consistency issues in func tests :-(21:16
*** jrosser has quit IRC21:29
*** jrosser has joined #openstack-swift21:32
*** biyiklioglu has quit IRC21:39
mattoliveraumorning21:44
*** diablo_rojo has quit IRC21:48
*** rcernin has joined #openstack-swift22:05
*** dsariel has quit IRC22:29
openstackgerritTim Burke proposed openstack/swift master: swift-init: Re-enable targeted configs  https://review.opendev.org/c/openstack/swift/+/77087122:45
openstackgerritTim Burke proposed openstack/swift master: swift-init: Include message when no executables found  https://review.opendev.org/c/openstack/swift/+/77087222:45
openstackgerritTim Burke proposed openstack/swift master: Monkey-patch in tests using mock.patch  https://review.opendev.org/c/openstack/swift/+/77087322:45
_david_sohonethttps://github.com/eventlet/eventlet/commit/9f49f0bbb9c18d7478d7c5e2e677ae6452845ab5, but I see you author23:01
_david_sohoneter a later patch on the same area of code: https://github.com/eventlet/eventlet/commit/af253ad97d6e33e86aca56ea5ce5ede1ed0a51cb . Should I take the first but not the second ?23:01
_david_sohonetSorry about the noise ^. I suck at IRC clearly..23:01
_david_sohonetclayg: you mentioned your proxy-servers were more stable with https://github.com/eventlet/eventlet/commit/9f49f0bbb9c18d7478d7c5e2e677ae6452845ab5, but I see you authorer a later patch on the same area of code: https://github.com/eventlet/eventlet/commit/af253ad97d6e33e86aca56ea5ce5ede1ed0a51cb . Should I take the first but not the second ?23:01
timburke__david_sohonet, you want both; they both came in as part of https://github.com/eventlet/eventlet/pull/64523:02
clayg👍 whatever is the latest eventlet is fine.23:03
_david_sohonetOk, thanks timburke_ . As an update on the deadlock issue, I pointed the bad app (ffmpeg - encoding video with a TempURL as the video source - lots and lots of 499s) at a canary proxy server and it seems to confirm the app is the issue.23:05
DHEthere's an official fix for the logging deadlock?23:05
timburke_no, not yet, sorry :-(23:05
DHEaww...23:05
timburke_i should really make another attempt at it, even if it's just to have the proxy-server turn off logging.logThreads or whatever during bring-up23:06
timburke__david_sohonet, do you have a good repro script? that'd be *amazing*! part of my trouble thus far has been how sporadically i'd see the issue in my (very low-traffic) home cluster23:07
DHEI don't know if it's worth trying to apply on other servers. while the proxy server is probably better behaved than the average client it's probably better to be safe23:07
timburke_does it usually just need to try re-encoding one (large-ish) object, or would i want to try it in a loop for a while?23:08
DHEmy workload is customer hardware downloading smallish (4 MB or so) objects. I believe there is an above average/high rate of users aborting transfers though23:11
openstackgerritTim Burke proposed openstack/swift master: Turn off logging.logThreads when monkey-patched  https://review.opendev.org/c/openstack/swift/+/75259323:25
timburke_DHE, _david_sohonet you guys might try ^^^ (and no stdlib changes) -- i'd be real interested in feedback on it!23:26
openstackgerritMerged openstack/swift master: Fix __exit__ calls  https://review.opendev.org/c/openstack/swift/+/77066623:51

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!