Wednesday, 2016-01-20

*** dshulyak_ has joined #openstack-solar08:00
*** salmon_ has joined #openstack-solar08:33
openstackgerritLukasz Oles proposed openstack/solar: Move all text from README to documentation  https://review.openstack.org/27006208:58
dshulyak_hello09:14
salmon_hi dshulyak_09:15
pigmejhello guys :)09:26
pigmejsalmon_: -1 for pbr again :D09:27
openstackgerritLukasz Oles proposed openstack/solar: Move all text from README to documentation  https://review.openstack.org/27006209:27
salmon_pigmej: pbr? where?09:27
pigmejbecause it requires README.md09:28
pigmej;p09:28
salmon_I reanamed it ti .rst09:28
pigmej2016-01-20 09:09:13.342 |     IOError: [Errno 2] No such file or directory: 'README.md'09:28
pigmej;P09:28
salmon_yeahm I changed setup.cfg now09:30
pigmej:)09:33
openstackgerritBogdan Dobrelya proposed openstack/solar: Fix 80 chars wraps for docs  https://review.openstack.org/27010010:11
pigmejdshulyak_: the problem is that afair we need to enforce all tables, all metadata etc BEFORE creating multiple clients10:12
pigmejI had similar fuckups with it but afair that was a solution (at least worked for me)10:13
pigmejBUT it's probably perfectly fine to use file based sqlite for tests :)10:13
salmon_I will test this one https://review.openstack.org/#/c/269018/10:18
pigmejcool :)10:20
pigmejsalmon, ideally you should test 2 in the same tim10:20
pigmejtime10:20
salmon_2?10:20
salmon_how?10:20
pigmejhttps://review.openstack.org/#/c/269238/ + https://review.openstack.org/#/c/269018/10:20
salmon_ah10:20
pigmejlock fixes just part of problems, the other part is fixed by counter10:21
salmon_ok10:21
salmon_and I need to change how celery starts?10:21
pigmejyou can test them separately BUT you will need to know what errors are caused by what :)10:21
pigmejrevert https://review.openstack.org/#/c/267769/1/utils/solar-celery10:21
salmon_ok10:22
pigmejsalmon_ + bogdando you should agree which patch will be merged first :)10:24
bogdandopigmej, I lost the context10:24
salmon_pigmej: bogdano patch was first10:25
pigmejhttps://review.openstack.org/#/c/270100/ vs https://review.openstack.org/#/c/269673/10:26
pigmej?10:26
salmon_yup10:26
bogdandoIMHO the https://review.openstack.org/#/c/270100/10:32
salmon_yup10:32
pigmejbogdando: thanks for patch :)10:34
bogdandonp, let's W+110:34
openstackgerritMerged openstack/solar: Fix 80 chars wraps for docs  https://review.openstack.org/27010010:39
pigmejsalmon_: how is testing going ?10:52
pigmejdshulyak_: you wanted to understand / check why lock is complicated now, isn't it?10:53
salmon_pigmej: it's going :)10:53
pigmejk10:54
salmon_riak example worked10:55
pigmejmake '+' then :)10:58
salmon_I will check all examples ;)11:00
dshulyak_pigmej: yes, i will just dig into +- thing after i will fix sqlite11:09
pigmejk11:10
openstackgerritBogdan Dobrelya proposed openstack/solar: Add tox checkniceness,checksyntax for docs  https://review.openstack.org/26955911:35
bogdandopigmej, salmon_  all works now11:35
bogdandono more unrequired deps installing11:35
bogdandoand main tests ain't affected at all11:36
pigmejbogdando: I still don't get why `envlist = pep8,py27,docs,checkniceness,checksyntax`11:37
bogdandothat is for default "tox" invocation11:37
bogdandowe want it to do 1) generic tests, 2) build docs, 3) check style for docs11:37
bogdandowhat looks most reasonable, at least for me...11:38
bogdandoas a dev you may use only -e py27,pep811:38
bogdandoand nothing will change for you11:39
pigmejyaeh but the question is do we really need "doc test" on default tox invocation?11:39
bogdandowhy not?11:39
bogdandotox shall build docs as well and check them. I can't get your point, why not?11:40
pigmejMaybe I'm too python oriented or too dev oriented11:41
pigmejbut I don't see a point of building  docs on 'tox' invocation11:41
pigmejbut it's my personal opinion :)11:41
pigmejand feeling.11:41
bogdandodevs impact: replace "tox" to "tox -e py27,pep8". users impact: instead tox -l, and consequent tox -e docs && tox -e checkniceness && tox -e checkstyle .. just tox!11:41
bogdandojust let's think about users and tech writers as well ;)11:42
pigmejwell11:42
pigmejif one *just* plays with docs, he for sure don't want our full test suite11:42
bogdandook, I can remove epy27,pep8 from envlist :D11:42
*** openstackgerrit has quit IRC11:43
bogdandobut I believe that is the most expected defautls for just "tox" to do all it can11:43
pigmejI think we should keep there py27,pep8 OR docs related things, not mixing them together :)11:43
bogdandoI don't agree, but if you insist...11:44
-openstackstatus- NOTICE: review.openstack.org is being restarted to apply patches11:44
*** ChanServ changes topic to "review.openstack.org is being restarted to apply patches"11:44
pigmejbogdando: well, dont' change it becaues I said it :)11:44
*** openstackgerrit has joined #openstack-solar11:44
pigmejlet's wait for other opininos :)11:44
salmon_pigmej: Tested with all our examples. All worked11:47
pigmejsalmon_: perfect, not that I expected something other ;P11:47
salmon_:D11:47
openstackgerritLukasz Oles proposed openstack/solar: Increase riak timeout - time depends on download speed  https://review.openstack.org/27014711:49
pigmejsalmon_: is there any other way to not base on this timeout but some dynamic polling?11:52
salmon_pigmej: we coud change it to "timeout for time since last change"11:53
salmon_but current timeout is quite ok and on CI they have faster net :)11:54
pigmejk11:54
pigmejfaster than mine too?11:54
*** ChanServ changes topic to "#openstack-solar"11:55
-openstackstatus- NOTICE: Restart done, review.openstack.org is available11:55
salmon_pigmej: no idea11:55
pigmej:P11:55
openstackgerritLukasz Oles proposed openstack/solar: Update Wordpress tutorial  https://review.openstack.org/26967311:59
salmon_bogdando: what is the link to the solar dashboard?12:36
salmon_on gerrit12:36
bogdandohttps://review.openstack.org/#/dashboard/?foreach=%28project%3Aopenstack%2Fsolar%29+status%3Aopen+NOT+label%3AWorkflow%3C%3D%2D1+NOT+label%3ACode%2DReview%3C%3D%2D2&title=Solar+Review+Inbox&My+Patches+Requiring+Attention=owner%3Aself+%28label%3AVerified%2D1+OR+label%3ACode%2DReview%2D1+OR+label%3AWorkflow%3C%3D%2D1%29&Needs+Approval=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%2D1+NOT12:37
bogdando+label%3AVerified%3C%3D%2D1+NOT+owner%3Aself+label%3ACode%2DReview%3E%3D2+NOT+label%3ACode%2DReview%3C%3D%2D1&Ready+for+Core+Reviewers=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%2D1+NOT+label%3AVerified%3C%3D%2D1+label%3AVerified%3E%3D1+label%3ACode%2DReview%3E%3D1+NOT+label%3ACode%2DReview%3E%3D2+NOT+label%3ACode%2DReview%3C%3D%2D1&5+Days+Without+Feedback=NOT+label%3AWorkflow%3E%3D12:37
bogdando1+NOT+label%3AWorkflow%3C%3D%2D1+NOT+label%3AVerified%3C%3D%2D1+NOT+label%3ACode%2DReview%3C%3D2+age%3A5d&No+Feedback=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%2D1+NOT+label%3AVerified%3C%3D%2D1+NOT+label%3ACode%2DReview%3C%3D2+NOT+age%3A5d+limit%3A50&Disagreement=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%2D1+NOT+label%3AVerified%3C%3D%2D1+label%3ACode%2DReview%3C%3D%212:37
bogdandoD1+label%3ACode%2DReview%3E%3D1&Negative+Feedback=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%2D1+NOT+label%3AVerified%3C%3D%2D1+label%3ACode%2DReview%2D1+NOT+label%3ACode%2DReview%3C%3D%2D2+NOT+label%3ACode%2DReview%3E%3D1+limit%3A2012:37
salmon_what a long url :D12:38
salmon_thx12:38
openstackgerritMerged openstack/solar: Update Wordpress tutorial  https://review.openstack.org/26967312:40
dshulyak_can be merged https://review.openstack.org/#/c/269238/ ?13:03
pigmejdshulyak_: I think so, it just misses workflow flag13:04
openstackgerritDmitry Shulyak proposed openstack/solar: Use isolated filebased sqlite for each test process  https://review.openstack.org/27018113:08
openstackgerritDmitry Shulyak proposed openstack/solar: Use isolated filebased sqlite for each test process  https://review.openstack.org/27018113:08
pigmejdshulyak_: why do you make gevent non optional ?13:10
pigmejI dislike that fact.13:10
pigmejalso that --clean sounds to me like bad idea.13:11
dshulyak_thats for ci environments13:11
dshulyak_why is it bad?13:11
pigmejbecause you don't check if it's sqlite13:11
pigmejand you will remove any file that matches 'database' name13:11
dshulyak_i check if thats file13:12
dshulyak_ah13:12
pigmejI can have 'riak' filename or any other that matches databasename13:12
pigmej:)13:12
pigmejdshulyak_: what about gevent ? why you moved it to dblayer back ?13:13
openstackgerritMerged openstack/solar: Fixing concurrency problems in history counter  https://review.openstack.org/26923813:13
dshulyak_no13:13
dshulyak_i moved only dblayer part13:13
dshulyak_i can wrap it in similar try/except13:13
dshulyak_and i moved it because it imports too much13:14
pigmejbut why you moved it there ?13:14
dshulyak_because it is dblayer patches?13:14
pigmejit also patches utils13:15
dshulyak_i need to remove them from solar/__init__.py13:15
dshulyak_and dblayer/__init__ seems to be the right place13:15
pigmejwe had it in dblayer before, and it turned out that it's too late to have it there13:15
dshulyak_no13:15
dshulyak_i remember that problem13:15
dshulyak_it wasnt related even :)13:15
pigmeju sure ?13:16
dshulyak_you jsut moved it :)13:16
pigmejwell, but then you need to move utils patch from there13:16
pigmejbecause it's possible that utils is not patched correctly13:16
dshulyak_hm, what they are doing in dblayer :)13:17
pigmejthat's another question :P13:18
dshulyak_should i patch them in utils?13:18
pigmejthey should be patched together with the rest of our patches13:18
pigmejbecause they depend13:19
pigmejbut if your intention was to just move that paches then ok, BUT it has to be conditional13:20
dshulyak_this 2 shouldnt depend on dbalyer13:21
dshulyak__patch(utils, 'get_local', get_local)13:21
dshulyak_    _patch(utils, 'get_current_ident', get_current_ident)13:21
pigmejbut they were needed only for dblyaer13:22
pigmejbut yeah13:22
pigmejthey shouldn't now probably13:22
dshulyak_ok, i will leave them in place for now, they wont be used wo importing db first anyway13:24
pigmejkinda13:24
pigmej:)13:24
openstackgerritDmitry Shulyak proposed openstack/solar: Use isolated filebased sqlite for each test process  https://review.openstack.org/27018113:24
pigmejhttps://github.com/openstack/solar/blob/master/solar/dblayer/gevent_patches.py#L29-L3513:25
pigmejdshulyak_: how this format will work when someone will test on riak ?13:25
dshulyak_format wont have any effect if there is no {PID} in url13:28
pigmejlet's assume that noone will abuse this :D13:28
pigmejdshulyak_: and with this everything works for you ?13:29
dshulyak_with file based sqlite tests for new worker are fine, i will rebase on top of this patch in a moment13:30
pigmejcool :)13:30
pigmejbtw can you test this locking, because it's now blocker for release13:30
openstackgerritDmitry Shulyak proposed openstack/solar: Zerorpc worker for orchestration modules  https://review.openstack.org/26916613:31
dshulyak_i will review in 20-30 mins13:41
pigmejperfect :)13:44
*** bogdando has quit IRC13:56
*** bogdando has joined #openstack-solar14:02
pigmejok guys, I think it's nonsense to "improve" errors in repository14:04
pigmej`OSError: [Errno 2] No such file or directory: '/var/lib/solar/repositories/resources/ratarstar/2.0.0'14:05
pigmej` this is better than any custom made error, isn't it?14:05
pigmejit's very easy to use it as API then (standard python errors), I can wrap the errors in some RepositoryException BUT... I'm not sure if it's correct idea14:05
pigmejsalmon_: dshulyak_ opinions ?14:05
pigmejIt will require a lot of work, with a lot of errno.E* things, and we will probably gain nothing except more pretty string error14:06
salmon_pigmej: depends when do you see this error. On what action?14:06
pigmej`solar repository` cli actions14:06
pigmejI could hide traceback in CLI, and print str in red14:07
pigmejthis one14:09
pigmejhttps://bpaste.net/show/3e695d20b69214:09
pigmejis also pretty clear14:10
salmon_good ux means not exceptions :)14:10
salmon_*no14:10
pigmejsalmon_: this one is python API usage => exception is ok14:10
salmon_I'm talking about cli14:11
pigmejhttps://bpaste.net/show/1c4b492f2d5214:11
pigmejwhat would you like to see in this case?14:11
salmon_https://bpaste.net/show/4774387e119114:13
pigmejI'm pretty sure that we should catch errors like that on CLI level14:13
salmon_yup14:13
pigmejso I'm not touching errors in repository then14:13
pigmejexcept one place where I need to consolidate things14:13
openstackgerritJedrzej Nowak proposed openstack/solar: Unified exception in repo create  https://review.openstack.org/27020814:15
salmon_for sure it should be only on CLI level14:15
dshulyak_pigmej: what is the handling for KeyError in who_is_locking and change_locking_start doing?14:26
pigmejdblayer :D14:27
dshulyak_i mean when it is possible14:27
pigmejwait I need to look into code :)14:27
pigmejyou mean that identity keyerror?14:28
pigmejit's needed because ensemble uses your 'old' method14:28
pigmejwhere you set `identity`14:28
pigmejso keyerror is possible when no identity is set14:30
pigmejare you talking about that KeyERror or other ?14:31
dshulyak_yes, i see14:31
pigmejIt's kinda fleaky api14:32
pigmejwe can simplify it later (we can use riak types on riak, and this implementation on sqlite), and get rid of this "identity" completely, or make special class for ensemble only etc14:34
dshulyak_then i think i understood how it works, even if 2 concurrent threads will change lockers with 1 - they will agree on what thread is erroneous because of sorting in sum_all14:36
pigmejthis too,14:37
pigmejbut with n_val=1 first will win, second will fail no matter of 'sorting' order. This implementation prevents storing "old" value after storing new14:37
pigmejso our "delete" case is no-go now :)14:38
pigmejhttps://bpaste.net/show/233e13105f2715:06
pigmejsalmon_: ^15:06
salmon_:)15:10
dshulyak_i think we should just remove traceback somewhere in click hooks15:12
pigmejdshulyak_: yeah working on it :)15:13
pigmej(see --debug) :)15:13
dshulyak_i missed discussion :)15:13
pigmejthough I created separate group15:13
pigmejbtw salmon_ , dshulyak_ one of you need set workflow flag :)15:20
pigmejhttps://review.openstack.org/#/c/269018/15:20
openstackgerritJedrzej Nowak proposed openstack/solar: Added EGroup click group which can handle errors  https://review.openstack.org/27024715:20
dshulyak_is it possible to set debug in main ?15:23
pigmejno but it could be possible15:24
pigmejgenrally click is stupid15:24
pigmejbecause you will need to call it15:24
pigmejsolar repository --debug import resources -l15:24
pigmejnot solar repository import resources -l --debug15:24
pigmejso with main it will be even worse15:25
pigmejsolar --debug repository import resources -l15:25
dshulyak_i think it is better to set solar —debug15:26
salmon_+115:26
pigmejk15:27
openstackgerritMerged openstack/solar: CRDTish lock to avoid concurrent update/delete  https://review.openstack.org/26901815:30
openstackgerritJedrzej Nowak proposed openstack/solar: Added EGroup click group which can handle errors  https://review.openstack.org/27024715:31
pigmejdshulyak_: salmon_ done15:31
openstackgerritDmitry Shulyak proposed openstack/solar: Use isolated filebased sqlite for each test process  https://review.openstack.org/27018116:18
openstackgerritDmitry Shulyak proposed openstack/solar: Zerorpc worker for orchestration modules  https://review.openstack.org/26916616:18
openstackgerritMerged openstack/solar: Increase riak timeout - time depends on download speed  https://review.openstack.org/27014716:28
openstackgerritBogdan Dobrelya proposed openstack/solar: Add tox checkniceness,checksyntax for docs  https://review.openstack.org/26955916:38
pigmejdshulyak_: can you remind me how to test new workers?17:18
dshulyak_solar-worker17:19
dshulyak_there is not init script right now, i’m just running it in foreground17:19
pigmejah right :)17:19
pigmejbtw salmon_ dshulyak_ I responded: https://review.openstack.org/#/c/269653/3 :)17:25
pigmejhmmmmmmmmmm, I'm starting really to dislike git, or I have some stupid defaults, I will update this PR with things that should be there.17:38
pigmej(I will fix that stderr to dshulyak_ )17:38
openstackgerritJedrzej Nowak proposed openstack/solar: Unify returned transport results  https://review.openstack.org/26965317:41
openstackgerritJedrzej Nowak proposed openstack/solar: Unify returned transport results  https://review.openstack.org/26965317:52
pigmejdshulyak_: good job with that new worker :)17:59
pigmejdshulyak_: but I have one question, could we later add some notification state about locking ? (in addition to this sleep) ?18:00
pigmejor at least we should randomize sleep,18:00
dshulyak_what kind of notification?18:00
pigmej"someone released lock" etc18:01
pigmejdshulyak_: hmm, I'm having a lot of siblings errors though18:01
dshulyak_i think it is not rebased after recent merges18:02
pigmejah18:02
dshulyak_or maybe it is :)18:02
pigmejthen bad18:02
pigmejhttps://bpaste.net/show/3d7b0a7ca6a418:02
dshulyak_yeah, i think it is rebased18:02
pigmejmaybe you somehow abuse cache ?18:03
pigmejbecause I don't see errors on first execution18:03
pigmejbut it kinda /always/ fails on second18:03
pigmejyeah, solar o restart => always error, but when I will restart worker first, and then solar o restart then everything is fine18:04
dshulyak_yeah, atleast it is reproducible :)18:06
dshulyak_should be east18:07
dshulyak_easy18:07
dshulyak_ah crap18:07
dshulyak_i broke session_start/session_end18:07
dshulyak_in my hooks mechanism i always pass context everywhere18:08
dshulyak_and session_start/session_end doesnt accept it18:08
dshulyak_there should be warning18:08
pigmej;D:D18:09
pigmejwell, it can't know wtf18:09
pigmejI mean how would you like to know that?18:12
dshulyak_sorry, to know what?18:13
pigmej"<dshulyak_> there should be warning"18:14
dshulyak_in logs there should be a warning that ModelMeta.session_start crashed18:14
dshulyak_actually it should be error18:14
dshulyak_hm, now i have another error :)18:15
dshulyak_SiblingsError: No empty object for resolution not sure what to do with siblings: ['{"status": "PENDING", "task_type": "solar_resource", "target": "5168ef2d4b1177d9726eaaca75d852d3", "args": ["hosts_file1", "run"], "childs": [], "parents": ["system_log:1545e573-270d-4e57-b6c0-14c22d867bc6~node2.run", "system_log:1545e573-270d-4e57-b6c0-14c22d867bc6~node1.run"], "execution": "system_log:1545e573-270d-4e57-b6c0-14c22d867bc6", "errmsg": "", "name":18:15
dshulyak_"hosts_file1.run"}', '{"status": "SUCCESS", "task_type": "solar_resource", "target": "5168ef2d4b1177d9726eaaca75d852d3", "args": ["hosts_file1", "run"], "childs": [], "parents": ["system_log:1545e573-270d-4e57-b6c0-14c22d867bc6~node2.run", "system_log:1545e573-270d-4e57-b6c0-14c22d867bc6~node1.run"], "execution": "system_log:1545e573-270d-4e57-b6c0-14c22d867bc6", "errmsg": "", "name": "hosts_file1.run"}']18:15
pigmejyeah i's conflict18:16
pigmejyou reset it18:16
*** salmon_1 has joined #openstack-solar18:17
dshulyak_you think thats the cache issue?18:18
pigmejyeah18:19
pigmejI never had it when I'm restarting worker after each execution18:20
*** bogdando has quit IRC18:22
*** salmon_ has quit IRC18:22
pigmejbtw what did you mean by context ?18:22
*** bogdando has joined #openstack-solar18:28
dshulyak_i information that is related to orchestration there18:32
pigmejaha18:33
dshulyak_strange.. i am certainly using session_end now18:34
pigmejit's always in the same greenlet ?18:35
pigmejmaybe you call session_end in incorrect greenlet?18:36
pigmejbut can you confirm that it works 'once' ?18:38
dshulyak_yeah, after reset it starts to throw errors18:40
pigmejyeah, but if you reset AND restart solar-worker, then everything is fine18:41
pigmejat least for me18:41
dshulyak_yes, the problem is still with session, im not calling it for some reason..18:43
pigmej"good" at least it's easy to debug:)18:44
openstackgerritDmitry Shulyak proposed openstack/solar: Zerorpc worker for orchestration modules  https://review.openstack.org/26916618:47
dshulyak_thanks, i was just stupid :) instead of calling function - i assigned it18:47
pigmej:D:D:D18:48
pigmejshit happens :D18:49
*** dshulyak_ has quit IRC19:22
*** dshulyak_ has joined #openstack-solar20:18
*** dshulyak_ has quit IRC20:28
*** dshulyak_ has joined #openstack-solar20:42
*** dshulyak_ has quit IRC20:53
*** salmon_1 has quit IRC22:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!