| ba8adaf2506d4bf2b9b09bf54f2490ced31dd16c |
|
05-Jul-2017 |
Prathmesh Prabhu <pprabhu@chromium.org> |
[autotest] Drop remaining references to autotest_stats
We are neither looking at the graphite dashboard anymore, not
maintaining it really. OTOH, the autotest_stats module costs ~.25
seconds on module load time, hitting us in places where it matters.
So, drop all uses, also removing the false sense of security.
BUG=chromium:739466
TEST=unittests.
Change-Id: I0e4118653d929a5d55f4ff62b96e9c66e855efd7
Reviewed-on: https://chromium-review.googlesource.com/559832
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 5d94b2b13f90e8016b548d64b353c0b179498466 |
|
23-Feb-2017 |
Dan Shi <dshi@google.com> |
[autotest] log rpc server name in the metadata logged to metadb
BUG=None
TEST=local test
Change-Id: Ic63b93ac5f8d19e8f00ffb2527e2ac8b9ca3878f
Reviewed-on: https://chromium-review.googlesource.com/446727
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Kevin Cheng <kevcheng@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 14cac44542bc6a403cb7a3585eb3ebae00227992 |
|
21-Nov-2016 |
Aviv Keshet <akeshet@chromium.org> |
autotest: move or eliminate a bunch of autotest_stats stats
BUG=chromium:667171
TEST=unittests; guado_moblab tryjob + test run
Change-Id: I815d2d748a2daa3bc869ead5d09f722245b7317b
Reviewed-on: https://chromium-review.googlesource.com/413103
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| e782e855a366eb4c9e51540c6e4d3440caf82772 |
|
30-Jun-2015 |
MK Ryu <mkryu@google.com> |
[autotest] Include client's IP in RPC logging
Having the RPC requester's IP address in the rpc log and in the
metaDB will help us a lot to debug lab fire.
BUG=chromium:505568
TEST=Request a RPC to local apache from a remote machine.
DEPLOY=apache
Change-Id: I283f55d78eb29ff59cce6e487661adaf84dea84a
Reviewed-on: https://chromium-review.googlesource.com/282574
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Paul Hobbs <phobbs@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 1e1c41b1b4a1b97c0b7086b8430856ed45e064d3 |
|
05-Feb-2015 |
Gabe Black <gabeblack@chromium.org> |
graphite: Separate out configuration from the statsd classes.
The new version of the statsd classes should be created using an instance of
the new Statsd class which sets up some defaults without having to specify
them over and over. This makes it essentially compatible with the existing
usage in autotest, but will allow chromite to configure things differently and
avoid having side effects from importing the module or global state.
BUG=chromium:446291
TEST=Ran unit tests, ran stats_es_functionaltest.py, ran the
stats_mock_unittest, ran a butterfly-paladin tryjob with --hwtest, testing by
fdeng.
DEPLOY=apache,scheduler,host-scheduler
Change-Id: I1071813db197c0e5e035b4d8db615030386f1c1c
Reviewed-on: https://chromium-review.googlesource.com/246428
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Gabe Black <gabeblack@chromium.org>
Tested-by: Gabe Black <gabeblack@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 9d33e7c57959d91d49e802ce198a01215c7c5868 |
|
16-Dec-2014 |
Dan Shi <dshi@chromium.org> |
[autotest] Log RPC parameters to metaDB
This is added to troubleshooting the cause of long RPC with logging the details
of the request.
BUG=chromium:442840
TEST=local rpc call, confirm metadata is logged:
http://172.25.61.45:9200/_plugin/elastic-hammer/
search for:
{
"query": {"bool": {"minimum_should_match": 1,
"should": [{"term": {"_type": "rpc"}}]}},
"size": 100,
"sort": [{"time_recorded": "desc"}]}
expect hits like:
_source{}
params[]
id: 1527
is_complete: true
subname: "dshi_mtv.rpc"
time_recorded: 1418767090.919642
method: "get_special_tasks"
value: 0.012034893035888672
id: "jsonrpc"
Change-Id: Id0f32fcbb621341efb5d17a07348ea887f58c6f7
Reviewed-on: https://chromium-review.googlesource.com/236074
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 59cfe5497d7f3be2bc4ade144d649a131e88448e |
|
03-Sep-2014 |
Jakob Juelich <jakobjuelich@chromium.org> |
[autotest] Add heartbeat AFE endpoint to shard Autotest
To improve the workload the autotest setup can handle, jobs should
be executed by multiple shards. They are sharded by type of board.
This a HTTP accessible endpoint to AFE. This assigns jobs and hosts
to moblabs and returns them as records to insert into a shard's
local database.
TEST=Ran suites and tried manually retrieving jobs
DEPLOY=scheduler,apache,host-scheduler
Change-Id: I20ac41364e4c67e17883729181af798e18d8093e
Reviewed-on: https://chromium-review.googlesource.com/217968
Reviewed-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 23b2895d2c3819a0191deab727327bcb894a0f99 |
|
13-Sep-2014 |
Fang Deng <fdeng@chromium.org> |
This is suspect to break our afe "debug" link. I am going to revert it for now.
Revert "[autotest] Add heartbeat AFE endpoint to shard Autotest"
This reverts commit 1e10d745c65ecafa79bd4f0b4f0b743bd5f1eff3.
Change-Id: I81fd554a910c8c5b7537b34ec13301fcf15be3fd
Reviewed-on: https://chromium-review.googlesource.com/217980
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 1e10d745c65ecafa79bd4f0b4f0b743bd5f1eff3 |
|
03-Sep-2014 |
Jakob Juelich <jakobjuelich@chromium.org> |
[autotest] Add heartbeat AFE endpoint to shard Autotest
To improve the workload the autotest setup can handle, jobs should
be executed by multiple shards. They are sharded by type of board.
This a HTTP accessible endpoint to AFE. This assigns jobs and hosts
to moblabs and returns them as records to insert into a shard's
local database.
TEST=Ran suites and tried manually retrieving jobs
DEPLOY=scheduler,apache,host-scheduler
Change-Id: Ie1dfa7bb295f685c56bb1354675d83f82d933c65
Reviewed-on: https://chromium-review.googlesource.com/211214
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| cd0a01d0017e6f4c8b5a83890021cefeae3ca64f |
|
12-Jun-2014 |
Dan Shi <dshi@chromium.org> |
[autotest] Let log socket server use an unused port
BUG=chromium:383370
TEST=unittest
Change-Id: I56df36cb230d4f4f81f8ce1525b417a132c15aaf
Reviewed-on: https://chromium-review.googlesource.com/203542
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| da8c60af1e1e3ee97170c700d0b72991687e35a2 |
|
03-Jun-2014 |
Michael Liang <michaelliang@chromium.org> |
[autotest] Migrate graphite directory to client/common_lib/cros
This change allows us to report stats in client tests.
1. Change import paths for all files that import modules from graphite
2. Clean up some unused modules
Related CL: https://chromium-review.googlesource.com/#/c/202467/
BUG=chromium:237255
TEST=Ran scheduler locally, scheduled reboot jobs, verified stats such as monitor_db_cleanup.user_cleanup._cleanup were reported on chromeos-stats.
DEPLOY = apache, scheduler, host_scheduler
Change-Id: Iebfe3b8acc1c363a0b70ea555744e85d1367cb67
Reviewed-on: https://chromium-review.googlesource.com/202727
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Michael Liang <michaelliang@chromium.org>
Tested-by: Michael Liang <michaelliang@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 0f5f0443e9286483365d6978ff432d8e4029d599 |
|
23-May-2014 |
Jiaxi Luo <jiaxiluo@chromium.org> |
[autotest] Timer on rpc handler
Leverage on statsd and graphite to monitor on rpc_interface
through rpc handler. This will give us a good idea of the
bottlenecks on DB latency and throughput, and reflect on
the changes made on DB schema.
There is already a counter at serviceHandler.py. The timer is added
in a similar manner.
BUG=None
TEST=Ran afe, ran cli, ran graphite
Change-Id: I4415aacc920db339b18d7208b841747843830997
Reviewed-on: https://chromium-review.googlesource.com/201381
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 1b2493186b54a7707e9e3c731021049d49932c75 |
|
13-Nov-2013 |
Alex Miller <milleral@chromium.org> |
[autotest] Move rpc stats down the stack to cover all attempted RPCs.
This will log all RPCs, afe and tko, and even ones that error out or
don't make sense. This should let us get a realistic view of the RPC
load on our systems.
TEST=restarted apache, saw stats on chromeos-stats/
DEPLOY=apache
BUG=None
Change-Id: I8223347e86b358fa97c44f956c656a380edb3c48
Reviewed-on: https://chromium-review.googlesource.com/176591
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 4f34170dbb70080a56de88dc13859b3a7ffffb0a |
|
25-Mar-2013 |
Alex Miller <milleral@chromium.org> |
[autotest] Use |json| instead of |simplejson|.
simplejson was moved into the library as of python 2.4. We're now at
python 2.6/2.7, and having to keep installing simplejson everywhere has
slowly turned into a hassle. Since we can just trivially drop/rename
it, let's do so.
The rpc_interface parts of this change have been vetted, the rest is
done just as a s/simplejson/json/ across the code, which afaik should be
a safe thing to do.
BUG=None
TEST=unit, run_suite
DEPLOY=apache
Change-Id: Iab35f71157e7e5cd3be5291c776b07eb3283be8e
Reviewed-on: https://gerrit.chromium.org/gerrit/46424
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 19841977ef4d476c48b7f62207f9c81183871f46 |
|
11-Jan-2010 |
mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> |
Remove the bundled version of simplejson on the autotest
tree, in order to be coherent with the general policy of
just requiring that the server admin install external
dependencies. For simplejson, it's not necessary to
maintain the module on our tree, since we are mostly
using a version very close to upstream 2.0.9.
Also, added simplejson to utils/build_externals.py, to
make installation straightforward in case one doesn't
want/can use the package shipped by the distro.
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@4088 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 2df3c69dfb02a69d64f93aeb6cc7887ca032ff27 |
|
11-Sep-2009 |
showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> |
fix jsonp_fetcher, it was broken by changes to ServiceHandler in previous patch to implement RPC logging
Signed-off-by: Steve Howard <showard@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@3706 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 02ed4bd47471fe77270e5507a7f9759955b3753c |
|
09-Sep-2009 |
showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> |
Reorganization of the host selection UI in the AFE create job view.
* refactor HostSelector into view/presenter (except for tables, which will need to be separately refactored)
* reorganize view into a tabbed format with a unified hostname input tab, a metahost tab and a browse hosts tab, with the selected hosts table alongside as before
Minor changes:
* extracted SimplifiedList interface from MultiListSelectPresenter to a top-level entity
* made ExtendedListBox implement the SimplifiedList interface and got rid of the SimplifiedListWrapper decorator class
* rewrote ArrayDataSource to use a TreeSet so it's actually efficient. this should speed up working with large host selections.
* edited JSONObjectComparator to be consistent with equals() for safety when using with TreeSet
* small change to how widths get set on TabPanels in AFE. rather than setting 100% width on all tab panels' contents using CSS, set it explicitly in CustomTabPanel. there's a quirk with browser layout where a 100% width div inside a table cell will have it's width set larger than the cell containing it. it happens on IE6, FF3 and Chrome so it's probably not something that's going to change. It does go away with the "border box" model, which can be enabled with nonstandard CSS attributes, but it's easy enough to work around here that that's not worth it.
Signed-off-by: Steve Howard <showard@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@3673 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| a5288b4bb2b09aafe914d0b7d5aab79a7e433eaf |
|
28-Jul-2009 |
showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> |
Upgrade from Django 0.96 to Django 1.0.2.
Risk: high (framework change)
Visibility: medium
Signed-off-by: James Ren <jamesren@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@3457 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 3d6ae118f69717e68bc15b9aed7b6a6c7dd9bab0 |
|
02-May-2009 |
showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> |
Export CSV support for spreadsheet view.
* new Django view, handle_csv(), with a new URLconf, /new_tko/server/csv/
* new module csv_encoder with logic to encode the results of an RPC call into CSV format
* logic in csv_encoder to convert results of get_status_counts() (or get_latest_tests()) into CSV, when called from spreadsheet view
* added optional "Export to CSV" link to TableActionsPanel, and made SpreadsheetView use it with a handler to call the /csv/ entry point with the current query info
Signed-off-by: Steve Howard <showard@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@3086 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| ef6fe028fcc667366e8ac30fe63ba314a4b1d745 |
|
27-Mar-2009 |
showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> |
generic JSON-RPC implementation using JSONP (JSON with Padding). the central part of this change consists of:
* refactored JsonRpcProxy to extract all XmlHttpRequest-specific logic into a new XhrHttpRequest subclass, and made JsonRpcProxy abstract
* introduced new PaddedJsonRpcProxy subclass of XmlHttpRequest that uses JSONP instead of XHR
* added new handle_jsonp_rpc_request() method to rpc_handler.py, to handle JSONP requests on the server side
This enables the entire frontend (either AFE or TKO) to operate via JSONP instead of XHR. I didn't make them do that now, since there's no reason, but it will be critical when we go to make components embeddable in other pages (on other domains). Other changes here include:
* made TestDetailView use PaddedJsonRpcProxy (it previous had its own custom JSONP logic, which was all moved into PaddedJsonRpcProxy).
* made retrieve_logs.cgi and jsonp_fetcher.cgi support JSONP requests, so that log fetching requests could go through the shared JsonRpcProxy codepath. retrieve_logs.cgi still supports the old GET params interface for backwards compatibility (at least old TKO still uses it, and possible other scripts).
Signed-off-by: Steve Howard <showard@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@2943 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| d876f459fff6cc4994cab329b1f80c99a86edcbd |
|
03-Dec-2008 |
mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> |
gps pointed out that "== and != work in most cases but its better to use is
and is not as you'll never run into a case where someone's __eq__ or __ne__
method do the wrong thing."
Signed-off-by: Martin J. Bligh <mbligh@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@2533 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 0afbb6369aa5aa9a75ea67dd9e95ec4b21c0c181 |
|
06-Jun-2008 |
jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> |
Convert all python code to use four-space indents instead of eight-space tabs.
Signed-off-by: John Admanski <jadmanski@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@1658 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 5da17d48573ce769a4bc94a228738fe4826af090 |
|
28-Apr-2008 |
showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> |
JSON service handler should return a traceback for this error path too
git-svn-id: http://test.kernel.org/svn/autotest/trunk@1470 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| 05fb6997f37ad8f62142be3092fc10ddb36433fb |
|
25-Mar-2008 |
showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> |
-On the server, when exceptions occur, pass back a traceback along with the other info
-On the client, added an expandable "error log" box at the bottom. When non-JSON responses are received, show the full response in the log. When exceptions are received, show the traceback in the log.
git-svn-id: http://test.kernel.org/svn/autotest/trunk@1361 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|
| e8819cdf80ca0e0602d22551a50f970aa68e108d |
|
15-Feb-2008 |
mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> |
Add frontend and scheduler for Autotest
The attached tarball includes the new Autotest web frontend for creating
and monitoring jobs and the new scheduler for executing jobs. We've
been working hard to get these complete and stabilized, and although
they still have a long, long way to go in terms of features, we believe
they are now stable and powerful enough to be useful.
The frontend consists of two parts, the server and the client. The
server is implemented in Python using the Django web framework, and is
contained under frontend/ and frontend/afe. The client is implemented
using Google Web Toolkit and is contained under frontend/client. We
tried to use Dojo initially, as was generally agreed on, but it proved
to be too young and poorly documented of a framework, and developing in
it was just too inefficient.
The scheduler is contained entirely in the scheduler/monitor_db Python
file. It looks at the database used by the web frontend and spawns
autoserv processes to run jobs, monitoring them and recording status.
The scheduler was developed by Paul Turner, who will be sending out some
detailed documentation of it soon.
I've also included the DB migrations code for which I recently submitted
a patch to the mailing list. I've included this code because it is
necessary to create the frontend DB, but it will (hopefully) soon be
part of the SVN repository.
Lastly, there is an apache directory containing configuration files for
running the web server through Apache.
I've put instructions for installing a full Autotest server, including
the web frontend, the scheduler, and the existing "tko" results backend,
at http://test.kernel.org/autotest/AutotestServerInstall. I've also
created a brief guide to using the web frontend, with plenty of
screenshots, at http://test.kernel.org/autotest/WebFrontendHowTo.
Please let me know if you find any problems with either of these pages.
Take a look, try it out, send me comments, complaints, and bugs, and I
hope you find it useful!
Steve Howard, and the rest of the Google Autotest team
From: Steve Howard <showard@google.com>
git-svn-id: http://test.kernel.org/svn/autotest/trunk@1242 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/frontend/afe/json_rpc/serviceHandler.py
|