kernel_optimize_test/tools/perf
Adrian Hunter 2987e32f75 perf script: Add Python script to export to postgresql
Add a Python script to export to a postgresql database.

The script requires the Python pyside module and the Qt PostgreSQL
driver.  The packages needed are probably named "python-pyside" and
"libqt4-sql-psql"

The caller of the script must be able to create postgresql databases.

The script takes the database name as a parameter.  The database and
database tables are created.  Data is written to flat files which are
then imported using SQL COPY FROM.

Example:

  $ perf record ls
  ...
  $ perf script report export-to-postgresql lsdb
  2014-02-14 10:55:38.631431 Creating database...
  2014-02-14 10:55:39.291958 Writing to intermediate files...
  2014-02-14 10:55:39.350280 Copying to database...
  2014-02-14 10:55:39.358536 Removing intermediate files...
  2014-02-14 10:55:39.358665 Adding primary keys
  2014-02-14 10:55:39.658697 Adding foreign keys
  2014-02-14 10:55:39.667412 Done
  $ psql lsdb
  lsdb-# \d
              List of relations
   Schema |      Name       | Type  | Owner
  --------+-----------------+-------+-------
   public | comm_threads    | table | acme
   public | comms           | table | acme
   public | dsos            | table | acme
   public | machines        | table | acme
   public | samples         | table | acme
   public | samples_view    | view  | acme
   public | selected_events | table | acme
   public | symbols         | table | acme
   public | threads         | table | acme
  (9 rows)
  lsdb-# \d samples
         Table "public.samples"
      Column     |  Type   | Modifiers
  ---------------+---------+-----------
   id            | bigint  | not null
   evsel_id      | bigint  |
   machine_id    | bigint  |
   thread_id     | bigint  |
   comm_id       | bigint  |
   dso_id        | bigint  |
   symbol_id     | bigint  |
   sym_offset    | bigint  |
   ip            | bigint  |
   time          | bigint  |
   cpu           | integer |
   to_dso_id     | bigint  |
   to_symbol_id  | bigint  |
   to_sym_offset | bigint  |
   to_ip         | bigint  |
   period        | bigint  |
   weight        | bigint  |
   transaction   | bigint  |
   data_src      | bigint  |
  Indexes:
      "samples_pkey" PRIMARY KEY, btree (id)
  Foreign-key constraints:
      "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id)
      "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id)
      "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id)
      "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id)
      "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id)
      "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id)
      "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id)
      "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id)

  lsdb-# \d samples_view
                 View "public.samples_view"
        Column       |          Type           | Modifiers
  -------------------+-------------------------+-----------
   id                | bigint                  |
   time              | bigint                  |
   cpu               | integer                 |
   pid               | integer                 |
   tid               | integer                 |
   command           | character varying(16)   |
   event             | character varying(80)   |
   ip_hex            | text                    |
   symbol            | character varying(2048) |
   sym_offset        | bigint                  |
   dso_short_name    | character varying(256)  |
   to_ip_hex         | text                    |
   to_symbol         | character varying(2048) |
   to_sym_offset     | bigint                  |
   to_dso_short_name | character varying(256)  |

    lsdb=# select * from samples_view;

   id| time       |cpu | pid  | tid  |command| event  |   ip_hex      |           symbol    |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name
   --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+----------
   1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe|    10 | [kernel]| 0       | unknown |         0| unknown
   2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe|    10 | [kernel]| 0       | unknown |         0| unknown
   3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe|    10 | [kernel]| 0       | unknown |         0| unknown

My notes (which may be out-of-date) on setting up postgresql so you can
create databases:

fedora:

        $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql
        $ sudo su - postgres -c initdb
        $ sudo service postgresql start
        $ sudo su - postgres
        $ createuser -s <your username>

I used the the unix user name in createuser.

If it fails, try createuser without -s and answer the following question
to allow your user to create tables:

        Shall the new role be a superuser? (y/n) y

ubuntu:

        $ sudo apt-get install postgresql
        $ sudo su - postgres
        $ createuser <your username>
        Shall the new role be a superuser? (y/n) y

You may want to disable automatic startup.  One way is to edit
/etc/postgresql/9.3/main/start.conf.  Another is to disable the init
script e.g. sudo update-rc.d postgresql disable

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-29 10:32:49 -02:00
..
arch perf tools: A thread's machine can be found via thread->mg->machine 2014-10-29 10:32:46 -02:00
bench perf bench futex: Sanitize -q option in requeue 2014-09-29 15:43:26 -03:00
config perf tools: Do not attempt to run perf-read-vdso32 if it wasn't built 2014-10-29 10:32:48 -02:00
Documentation perf script: Add period data column 2014-10-17 15:21:30 -03:00
python perf python: Remove duplicate TID bit from mask 2013-08-07 17:35:25 -03:00
scripts perf script: Add Python script to export to postgresql 2014-10-29 10:32:49 -02:00
tests perf tests: Use thread->mg->machine 2014-10-29 10:32:47 -02:00
ui perf top: Add a visual cue for toggle zeroing of samples 2014-10-15 15:47:48 -03:00
util perf scripting python: Extend interface to export data in a database-friendly way 2014-10-29 10:32:49 -02:00
.gitignore perf tools: Add perf-with-kcore script 2014-09-17 17:08:08 -03:00
builtin-annotate.c perf tools: Remove hists from evsel 2014-10-14 17:32:52 -03:00
builtin-bench.c perf bench: Add --repeat option 2014-06-19 16:13:15 -03:00
builtin-buildid-cache.c perf buildid-cache: Use strerror_r instead of strerror 2014-08-15 13:07:59 -03:00
builtin-buildid-list.c perf session: Separating data file properties from session 2013-10-21 17:33:25 -03:00
builtin-diff.c perf diff: Add missing hists__init() call at tool start 2014-10-22 16:31:06 -03:00
builtin-evlist.c perf tools: Modify error code for when perf_session__new() fails 2014-09-26 12:32:58 -03:00
builtin-help.c perf help: Use strerror_r instead of strerror 2014-08-15 13:08:26 -03:00
builtin-inject.c perf tools: A thread's machine can be found via thread->mg->machine 2014-10-29 10:32:46 -02:00
builtin-kmem.c perf tools: Modify error code for when perf_session__new() fails 2014-09-26 12:32:58 -03:00
builtin-kvm.c perf kvm: Print kvm specific --help output 2014-10-29 10:32:47 -02:00
builtin-list.c perf list: Add usage 2013-11-05 14:26:41 -03:00
builtin-lock.c perf tools: Modify error code for when perf_session__new() fails 2014-09-26 12:32:58 -03:00
builtin-mem.c perf tools: Modify error code for when perf_session__new() fails 2014-09-26 12:32:58 -03:00
builtin-probe.c perf probe: Use PARSE_OPT_EXCLUSIVE flag 2014-10-29 10:32:47 -02:00
builtin-record.c perf tools: Export usage string and option table of perf record 2014-10-29 10:32:47 -02:00
builtin-report.c perf report: Set callchain_param.record_mode for future use 2014-10-15 16:20:03 -03:00
builtin-sched.c perf sched: Stop updating hists stats, not used 2014-10-09 11:46:35 -03:00
builtin-script.c perf tools: Export usage string and option table of perf record 2014-10-29 10:32:47 -02:00
builtin-stat.c perf evsel: No need to drag util/cgroup.h 2014-10-17 12:17:40 -03:00
builtin-timechart.c perf tools: Export usage string and option table of perf record 2014-10-29 10:32:47 -02:00
builtin-top.c perf tools: Remove hists from evsel 2014-10-14 17:32:52 -03:00
builtin-trace.c perf tools: A thread's machine can be found via thread->mg->machine 2014-10-29 10:32:46 -02:00
builtin.h perf tools: Add new mem command for memory access profiling 2013-04-01 12:21:44 -03:00
command-list.txt perf tools: Add new mem command for memory access profiling 2013-04-01 12:21:44 -03:00
CREDITS
design.txt perf tools: Update some code references in design.txt 2014-03-18 18:17:06 -03:00
Makefile perf tools: Add 'build-test' make target 2014-01-16 16:26:26 -03:00
Makefile.perf perf tools: Add facility to export data in database-friendly way 2014-10-29 10:32:49 -02:00
MANIFEST perf kvm: Add stat support on s390 2014-07-16 17:57:33 -03:00
perf-archive.sh
perf-completion.sh perf sched: Introduce --list-cmds for use by scripts 2014-04-16 17:16:05 +02:00
perf-read-vdso.c perf tools: Build programs to copy 32-bit compatibility 2014-10-29 10:32:48 -02:00
perf-sys.h perf tools: Make CPUINFO_PROC an array to support different kernel versions 2014-10-29 10:27:36 -02:00
perf-with-kcore.sh perf tools: Add perf-with-kcore script 2014-09-17 17:08:08 -03:00
perf.c perf: Use strerror_r instead of strerror 2014-08-15 10:54:29 -03:00
perf.h perf tools: Export usage string and option table of perf record 2014-10-29 10:32:47 -02:00