00:00:44 #startmeeting CongressTeamMeeting 00:00:45 Meeting started Thu Aug 20 00:00:44 2015 UTC and is due to finish in 60 minutes. The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot. 00:00:46 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 00:00:49 The meeting name has been set to 'congressteammeeting' 00:00:55 hi thinrichs 00:01:23 Hi RuiChen 00:02:16 I have just a couple of technical things on the agenda: 00:02:29 1. Discuss tables with differing numbers of columns 00:02:35 2. Gating on tempest tests 00:02:46 3. Progress on distributed architecture 00:02:51 Is there anything else? 00:04:24 If anything comes to mind, pipe up. 00:04:30 if you have anything need my help, please feel free to assign to me :0 00:04:34 :) 00:05:10 RuiChen: thanks! Some of the blueprints for the distributed architecture are unclaimed. 00:05:27 yeah, I see it 00:05:29 #link https://blueprints.launchpad.net/congress 00:06:08 Not as many as I had thought though. 00:06:47 Let's start with discussing this code change: 00:06:51 #link https://review.openstack.org/#/c/213283/ 00:07:00 :) 00:07:13 Yingxin1: want to describe the problem it's solving? 00:07:22 yes 00:08:17 A table is used to generate other tables, report errors, and execute actions. 00:09:43 But I found that a table with multiple columns fail to do either of them. 00:10:18 And currently there are many bugs supporting tables with multiple columns 00:10:29 So I pushed this patch. 00:11:23 Right. The only main reason we've been hesitant to eliminate multiple columns for a single table is error. 00:11:32 The error table, that is. 00:11:45 It's always been useful for us to write the rule.. 00:11:50 error(vm) :- blah 00:11:54 and also write the rule... 00:12:00 error(vm, net) :- blah 00:12:40 I asked Murano if they were using multi-arity tables, and they said no. 00:12:51 So there's no customer we know of that this would break. 00:13:32 Yes, but an error table collecting many errors, it should also tell where these errors come from. 00:13:39 In the long run, we want to enable people to write different kinds of error statements. 00:14:08 Yingxin1: could you explain what you mean by "where these errors come from"? 00:14:13 Maybe with an example. 00:15:49 When there are many rules generating errors to an 'error()' table. 00:16:06 So there will be many kinds of error in a single error table. 00:16:55 But a single error table cannot tell what kind of error it is. 00:17:08 So it confuses users. 00:18:11 I don't think that having multiple error tables will be enough to explain where the error came from, but it might help classify the type of error. 00:18:29 It sounds like you want multiple error tables. 00:19:00 When someone asks for all the policy violations, we still need a way of displaying all those error tables. 00:19:25 That is, having multiple tables doesn't change the work we need to do at the CLI/Horizon layer. 00:19:50 It just enables us to add a syntax check. 00:20:25 I think in the long run, I'd lean toward turning 'error' into a modal operator. 00:20:30 So we'd write something like… 00:20:38 error[table1(vm)] :- blah 00:20:47 error[table2(vm, net)] :- blah 00:20:57 Then someone can ask for all the errors with a query like: 00:21:07 give me all x such that error[x] is true. 00:22:00 What do we all think? 00:22:06 Yes, error table can be a special one. 00:22:35 how we can get the different error tables row ? 00:22:42 congress policy row list chenrui_p error ? 00:23:23 With Yingxin1's change, you'd ask multiple queries: 00:23:32 congress policy row list chenrui_p error1 00:23:38 congress policy row list chenrui_p error2 00:24:13 Another change would have a special CLI command, e.g. 00:24:23 Or the error tables can have a naming convention 00:24:40 congress policy row list modal chenrui_p error 00:24:52 Congress can search for them automatically. 00:25:01 Yingxin1: you still need the special API call to do that. 00:25:32 I'm proposing a clean way to implement that API call and to write the policy rules. 00:26:28 I think so. 00:27:06 RuiChen: any objection to eliminating differing column-numbers for a single table? 00:27:17 It's time to move on. 00:27:56 no, but I think maybe we need a specs to clarify it? 00:28:57 I don't think we need a spec. The code is already written. 00:29:12 It's more a usability question than a technical question. 00:29:56 Let's follow up on comments on the code itself. 00:30:06 #topic Gating on Tempest tests 00:30:23 I've been trying to get the tempest tests working so we can add them to our gate. 00:30:33 Ensuring no code gets merged unless it passes the tempest tests. 00:30:58 This came up again this week when Congress wouldn't start in devstack, which the Murano team noticed. 00:31:20 The problem seems to be that the tempest tests fail > 50% of the time. 00:31:28 But it's a different test that fails each time. 00:31:30 Here's the latest. 00:31:45 http://logs.openstack.org/27/214327/5/experimental/gate-congress-dsvm-api/95cf8cd/console.html 00:32:04 Search for: Failed 2 tests 00:32:31 I've tried upping the timeouts and disabling tests, just so we can get basic devstack integration tested on each checkin. 00:32:40 But that hasn't worked out so well. 00:33:54 At this point, I'm thinking about just disabling all but a couple of tests that work reliably 00:34:15 and gating on those. 00:34:25 Then we can fix the flaky tests later. 00:34:31 Thoughts? 00:34:39 +1 for that, even if the tests are trivial 00:36:11 pballand: I saw your comment; do you know SKIPtest() works? 00:37:03 I don’t - I just figured if you were renaming methods, using “SKIP” instead of XXX would be better 00:37:28 I second thinrichs ’s plan. 00:37:32 Oh—got it. Thought you were saying I was using the wrong method name. 00:37:47 Will do. Seems tempest doesn't like us to skip tests. 00:37:49 #link http://docs.openstack.org/developer/tempest/HACKING.html#skipping-tests 00:38:45 Related question for the group: anyone know how to stop tempest from running entire files of tests? 00:39:13 B/c we have nova, neutron, cinder, swift, etc. running in devstack, tempest is running all of the tests it has for all of those services too. 00:39:18 Instead of just the Congress tests. 00:40:39 thinrichs: We can provide a file with the list of tests to run to tempest 00:40:40 maybe we can remove the file name prefix 'test***' 00:41:06 Sayaji: any docs on that? Can we do it in the gate pipeline? 00:41:55 BTW: found the proper way to skip tests in tempest: 00:41:56 #link http://docs.openstack.org/developer/tempest/HACKING.html#test-skips-because-of-known-bugs 00:42:09 thinrichs: I have to look it up 00:42:11 TLDR: Use the @skip_because decorator 00:42:32 Sayaji: could you look it up and send us a link (email or whatever)? 00:42:47 thinrichs: Sure, will do that 00:43:12 Sayaji: thanks! 00:43:24 thinrichs: np 00:43:34 Next topic: status reports. 00:43:36 #topic status 00:43:56 Mainly on the distributed architecture, but anything the group needs to discuss is fine too. 00:44:15 Since we're short on time, just volunteer if you have something. 00:44:47 I checked in some changes required for rule synchronization. 00:45:05 I pushed a draft of the dist-cross-process-dse spec. Design is still in progress, but it would be helpful if people could check the requirements 00:45:16 I started addressing some comments for the startup script. 00:45:20 s/requirements/Problem description/ 00:45:22 That’s all. 00:45:22 alexsyip: was that the change the eliminated duplicates from the DB? 00:45:28 Yes 00:45:33 Tim: I take your bug https://bugs.launchpad.net/congress/+bug/1486246 Neutron and HA tempest tests broken 00:45:33 Launchpad bug 1486246 in congress "Neutron and HA tempest tests broken" [High,New] - Assigned to Zhenzan Zhou (zhenzan-zhou) 00:45:52 alexsyip: I had to revert it because it wouldn't work with MySql 00:46:02 do you have detail logs? 00:46:14 mysql doesn’t support unique? 00:46:27 alexsyip: mysql requires the field to have a fixed width for it to be unique 00:46:39 And I think it could be at most 255 chars long. 00:46:50 (I could be wrong about that one though.) 00:46:57 That’s unfortunate. 00:47:21 alexsyip: yep. Also, we need to write a DB migration script every time we change the schema. 00:47:33 Turns out to be just a matter of running a script. 00:47:39 I had forgotten about it entirely. 00:48:03 #link https://github.com/openstack/congress/tree/master/congress/db/migration 00:48:27 zhenzanz: turns out it's not just the neutron tests. 00:48:35 zhenzanz: there are a bunch that nondeterministically fail 00:48:55 zhenzanz: see above discussion about tempest tests and gate. 00:49:26 pballand: cool! 00:49:32 ok, thanks. Sorry I'm late 00:49:46 #link https://review.openstack.org/#/c/214893/1/specs/liberty/dist-cross-process-dse 00:50:54 Let's all make a point of getting pballand comments on the spec over the next few days. 00:51:02 #action all review dist-cross-process-dse spec 00:51:34 zhenzanz: no worries. Figured you had missed that bit of the meeting. 00:51:38 thinrichs: thanks - unfortunately there is no full design there, but I wanted feedback sooner than later to make sure we are in agreement on the problem we are solving 00:52:03 pballand: sounds like a good idea to me. 00:52:30 pballand: remember that we already have broad consensus for the approach 00:52:45 pballand: so I wouldn't be timid about getting it out there. 00:53:00 sure 00:53:25 I am working on improving my DseNode skeleton, and updating the design incrementally 00:53:47 pballand: nice! I think a lot will become clearer once we have a first cut of that. 00:53:55 That brings to mind… 00:54:27 I was wondering if it's worth debugging the RPC functionality that we're adding *before* moving to the fully distributed architecture. 00:54:46 That is, we could get everything working over RPC using the existing DSE 00:54:46 what do you mean by debugging? 00:55:04 and then once we move to the RPC of oslo.messaging, at least we've eliminated some basic problems. 00:56:08 So we'd be sending messages over the deepsix messaging functionality. 00:56:44 Instead of reaching into the policy engines and datasources directly when implementing the API. 00:57:11 Or would that be too much work? 00:57:23 We already have an implementation of RPC using deepsix (though we abandoned it). 00:57:33 We could resurrect it. 00:58:37 not sure… 00:58:49 2 minutes left. 00:58:52 Anything else? 00:59:03 if we use the same rpc method name, though, it may make the transition easier 00:59:17 pballand: agreed. 01:00:04 It'll help us make sure we've removed all references to external services and singletons. 01:00:10 Times up. 01:00:31 Let's keep making good progress! 01:00:35 Bye 01:00:38 #endmeeting