1. 14 Mar, 2017 1 commit
    • Jitendra Kolhe's avatar
      mem-prealloc: reduce large guest start-up and migration time. · 1e356fc1
      Jitendra Kolhe authored
      Using "-mem-prealloc" option for a large guest leads to higher guest
      start-up and migration time. This is because with "-mem-prealloc" option
      qemu tries to map every guest page (create address translations), and
      make sure the pages are available during runtime. virsh/libvirt by
      default, seems to use "-mem-prealloc" option in case the guest is
      configured to use huge pages. The patch tries to map all guest pages
      simultaneously by spawning multiple threads. Currently limiting the
      change to QEMU library functions on POSIX compliant host only, as we are
      not sure if the problem exists on win32. Below are some stats with
      "-mem-prealloc" option for guest configured to use huge pages.
      
      ------------------------------------------------------------------------
      Idle Guest      | Start-up time | Migration time
      ------------------------------------------------------------------------
      Guest stats with 2M HugePage usage - single threaded (existing code)
      ------------------------------------------------------------------------
      64 Core - 4TB   | 54m11.796s    | 75m43.843s
      64 Core - 1TB   | 8m56.576s     | 14m29.049s
      64 Core - 256GB | 2m11.245s     | 3m26.598s
      ------------------------------------------------------------------------
      Guest stats with 2M HugePage usage - map guest pages using 8 threads
      ------------------------------------------------------------------------
      64 Core - 4TB   | 5m1.027s      | 34m10.565s
      64 Core - 1TB   | 1m10.366s     | 8m28.188s
      64 Core - 256GB | 0m19.040s     | 2m10.148s
      -----------------------------------------------------------------------
      Guest stats with 2M HugePage usage - map guest pages using 16 threads
      -----------------------------------------------------------------------
      64 Core - 4TB   | 1m58.970s     | 31m43.400s
      64 Core - 1TB   | 0m39.885s     | 7m55.289s
      64 Core - 256GB | 0m11.960s     | 2m0.135s
      -----------------------------------------------------------------------
      
      Changed in v2:
       - modify number of memset threads spawned to min(smp_cpus, 16).
       - removed 64GB memory restriction for spawning memset threads.
      
      Changed in v3:
       - limit number of threads spawned based on
         min(sysconf(_SC_NPROCESSORS_ONLN), 16, smp_cpus)
       - implement memset thread specific siglongjmp in SIGBUS signal_handler.
      
      Changed in v4
       - remove sigsetjmp/siglongjmp and SIGBUS unblock/block for main thread
         as main thread no longer touches any pages.
       - simplify code my returning memset_thread_failed status from
         touch_all_pages.
      Signed-off-by: 's avatarJitendra Kolhe <jitendra.kolhe@hpe.com>
      Message-Id: <1487907103-32350-1-git-send-email-jitendra.kolhe@hpe.com>
      Signed-off-by: 's avatarPaolo Bonzini <pbonzini@redhat.com>
      1e356fc1
  2. 12 Jan, 2017 1 commit
  3. 17 Oct, 2016 1 commit
  4. 02 Aug, 2016 1 commit
  5. 13 Jul, 2016 2 commits
  6. 22 Mar, 2016 1 commit
    • Markus Armbruster's avatar
      include/qemu/osdep.h: Don't include qapi/error.h · da34e65c
      Markus Armbruster authored
      Commit 57cb38b3 included qapi/error.h into qemu/osdep.h to get the
      Error typedef.  Since then, we've moved to include qemu/osdep.h
      everywhere.  Its file comment explains: "To avoid getting into
      possible circular include dependencies, this file should not include
      any other QEMU headers, with the exceptions of config-host.h,
      compiler.h, os-posix.h and os-win32.h, all of which are doing a
      similar job to this file and are under similar constraints."
      qapi/error.h doesn't do a similar job, and it doesn't adhere to
      similar constraints: it includes qapi-types.h.  That's in excess of
      100KiB of crap most .c files don't actually need.
      
      Add the typedef to qemu/typedefs.h, and include that instead of
      qapi/error.h.  Include qapi/error.h in .c files that need it and don't
      get it now.  Include qapi-types.h in qom/object.h for uint16List.
      
      Update scripts/clean-includes accordingly.  Update it further to match
      reality: replace config.h by config-target.h, add sysemu/os-posix.h,
      sysemu/os-win32.h.  Update the list of includes in the qemu/osdep.h
      comment quoted above similarly.
      
      This reduces the number of objects depending on qapi/error.h from "all
      of them" to less than a third.  Unfortunately, the number depending on
      qapi-types.h shrinks only a little.  More work is needed for that one.
      Signed-off-by: 's avatarMarkus Armbruster <armbru@redhat.com>
      [Fix compilation without the spice devel packages. - Paolo]
      Signed-off-by: 's avatarPaolo Bonzini <pbonzini@redhat.com>
      da34e65c
  7. 08 Feb, 2016 2 commits
    • Eric Blake's avatar
      qom: Swap 'name' next to visitor in ObjectPropertyAccessor · d7bce999
      Eric Blake authored
      Similar to the previous patch, it's nice to have all functions
      in the tree that involve a visitor and a name for conversion to
      or from QAPI to consistently stick the 'name' parameter next
      to the Visitor parameter.
      
      Done by manually changing include/qom/object.h and qom/object.c,
      then running this Coccinelle script and touching up the fallout
      (Coccinelle insisted on adding some trailing whitespace).
      
          @ rule1 @
          identifier fn;
          typedef Object, Visitor, Error;
          identifier obj, v, opaque, name, errp;
          @@
           void fn
          - (Object *obj, Visitor *v, void *opaque, const char *name,
          + (Object *obj, Visitor *v, const char *name, void *opaque,
             Error **errp) { ... }
      
          @@
          identifier rule1.fn;
          expression obj, v, opaque, name, errp;
          @@
           fn(obj, v,
          -   opaque, name,
          +   name, opaque,
              errp)
      Signed-off-by: 's avatarEric Blake <eblake@redhat.com>
      Reviewed-by: 's avatarMarc-André Lureau <marcandre.lureau@redhat.com>
      Message-Id: <1454075341-13658-20-git-send-email-eblake@redhat.com>
      Signed-off-by: 's avatarMarkus Armbruster <armbru@redhat.com>
      d7bce999
    • Eric Blake's avatar
      qapi: Swap visit_* arguments for consistent 'name' placement · 51e72bc1
      Eric Blake authored
      JSON uses "name":value, but many of our visitor interfaces were
      called with visit_type_FOO(v, &value, name, errp).  This can be
      a bit confusing to have to mentally swap the parameter order to
      match JSON order.  It's particularly bad for visit_start_struct(),
      where the 'name' parameter is smack in the middle of the
      otherwise-related group of 'obj, kind, size' parameters! It's
      time to do a global swap of the parameter ordering, so that the
      'name' parameter is always immediately after the Visitor argument.
      
      Additional reason in favor of the swap: the existing include/qjson.h
      prefers listing 'name' first in json_prop_*(), and I have plans to
      unify that file with the qapi visitors; listing 'name' first in
      qapi will minimize churn to the (admittedly few) qjson.h clients.
      
      Later patches will then fix docs, object.h, visitor-impl.h, and
      those clients to match.
      
      Done by first patching scripts/qapi*.py by hand to make generated
      files do what I want, then by running the following Coccinelle
      script to affect the rest of the code base:
       $ spatch --sp-file script `git grep -l '\bvisit_' -- '**/*.[ch]'`
      I then had to apply some touchups (Coccinelle insisted on TAB
      indentation in visitor.h, and botched the signature of
      visit_type_enum() by rewriting 'const char *const strings[]' to
      the syntactically invalid 'const char*const[] strings').  The
      movement of parameters is sufficient to provoke compiler errors
      if any callers were missed.
      
          // Part 1: Swap declaration order
          @@
          type TV, TErr, TObj, T1, T2;
          identifier OBJ, ARG1, ARG2;
          @@
           void visit_start_struct
          -(TV v, TObj OBJ, T1 ARG1, const char *name, T2 ARG2, TErr errp)
          +(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
           { ... }
      
          @@
          type bool, TV, T1;
          identifier ARG1;
          @@
           bool visit_optional
          -(TV v, T1 ARG1, const char *name)
          +(TV v, const char *name, T1 ARG1)
           { ... }
      
          @@
          type TV, TErr, TObj, T1;
          identifier OBJ, ARG1;
          @@
           void visit_get_next_type
          -(TV v, TObj OBJ, T1 ARG1, const char *name, TErr errp)
          +(TV v, const char *name, TObj OBJ, T1 ARG1, TErr errp)
           { ... }
      
          @@
          type TV, TErr, TObj, T1, T2;
          identifier OBJ, ARG1, ARG2;
          @@
           void visit_type_enum
          -(TV v, TObj OBJ, T1 ARG1, T2 ARG2, const char *name, TErr errp)
          +(TV v, const char *name, TObj OBJ, T1 ARG1, T2 ARG2, TErr errp)
           { ... }
      
          @@
          type TV, TErr, TObj;
          identifier OBJ;
          identifier VISIT_TYPE =~ "^visit_type_";
          @@
           void VISIT_TYPE
          -(TV v, TObj OBJ, const char *name, TErr errp)
          +(TV v, const char *name, TObj OBJ, TErr errp)
           { ... }
      
          // Part 2: swap caller order
          @@
          expression V, NAME, OBJ, ARG1, ARG2, ERR;
          identifier VISIT_TYPE =~ "^visit_type_";
          @@
          (
          -visit_start_struct(V, OBJ, ARG1, NAME, ARG2, ERR)
          +visit_start_struct(V, NAME, OBJ, ARG1, ARG2, ERR)
          |
          -visit_optional(V, ARG1, NAME)
          +visit_optional(V, NAME, ARG1)
          |
          -visit_get_next_type(V, OBJ, ARG1, NAME, ERR)
          +visit_get_next_type(V, NAME, OBJ, ARG1, ERR)
          |
          -visit_type_enum(V, OBJ, ARG1, ARG2, NAME, ERR)
          +visit_type_enum(V, NAME, OBJ, ARG1, ARG2, ERR)
          |
          -VISIT_TYPE(V, OBJ, NAME, ERR)
          +VISIT_TYPE(V, NAME, OBJ, ERR)
          )
      Signed-off-by: 's avatarEric Blake <eblake@redhat.com>
      Reviewed-by: 's avatarMarc-André Lureau <marcandre.lureau@redhat.com>
      Message-Id: <1454075341-13658-19-git-send-email-eblake@redhat.com>
      Signed-off-by: 's avatarMarkus Armbruster <armbru@redhat.com>
      51e72bc1
  8. 04 Feb, 2016 1 commit
  9. 23 Nov, 2015 1 commit
  10. 22 Jul, 2015 1 commit
    • Eduardo Habkost's avatar
      hostmem: Fix qemu_opt_get_bool() crash in host_memory_backend_init() · 6b269967
      Eduardo Habkost authored
      This fixes the following crash, introduced by commit
      49d2e648:
      
        $ gdb --args qemu-system-x86_64 -machine pc,mem-merge=off -object memory-backend-ram,id=ram-node0,size=1024
        [...]
        Program received signal SIGABRT, Aborted.
        (gdb) bt
        #0  0x00007ffff253b8c7 in raise () at /lib64/libc.so.6
        #1  0x00007ffff253d52a in abort () at /lib64/libc.so.6
        #2  0x00007ffff253446d in __assert_fail_base () at /lib64/libc.so.6
        #3  0x00007ffff2534522 in  () at /lib64/libc.so.6
        #4  0x00005555558bb80a in qemu_opt_get_bool_helper (opts=0x55555621b650, name=name@entry=0x5555558ec922 "mem-merge", defval=defval@entry=true, del=del@entry=false) at qemu/util/qemu-option.c:388
        #5  0x00005555558bbb5a in qemu_opt_get_bool (opts=<optimized out>, name=name@entry=0x5555558ec922 "mem-merge", defval=defval@entry=true) at qemu/util/qemu-option.c:398
        #6  0x0000555555720a24 in host_memory_backend_init (obj=0x5555562ac970) at qemu/backends/hostmem.c:226
      
      Instead of using qemu_opt_get_bool(), that didn't work with
      qemu_machine_opts for a long time, we can use the corresponding
      MachineState fields.
      Reviewed-by: 's avatarMarcel Apfelbaum <marcel@redhat.com>
      Signed-off-by: 's avatarEduardo Habkost <ehabkost@redhat.com>
      6b269967
  11. 22 Jun, 2015 1 commit
  12. 19 Jun, 2015 2 commits
  13. 01 Apr, 2015 1 commit
    • Lin Ma's avatar
      hostmem: Prevent removing an in-use memory backend · 36bce5ca
      Lin Ma authored
      showing a memory device whose memdev is removed leads an assert:
      
      (qemu) object_add memory-backend-ram,id=ram0,size=128M
      (qemu) device_add pc-dimm,id=d0,memdev=ram0
      (qemu) object_del ram0
      (qemu) info memory-devices
      **
      ERROR:qom/object.c:1274:object_get_canonical_path_component:\
                                  assertion failed: (obj->parent != NULL)
      Aborted
      
      The patch prevents removing an in-use mem backend and error out.
      Signed-off-by: 's avatarLin Ma <lma@suse.com>
      Message-Id: <1427704589-7688-3-git-send-email-lma@suse.com>
      Reviewed-by: 's avatarIgor Mammedov <imammedo@redhat.com>
      Signed-off-by: 's avatarPaolo Bonzini <pbonzini@redhat.com>
      36bce5ca
  14. 20 Aug, 2014 1 commit
    • Michael S. Tsirkin's avatar
      hostmem: set MPOL_MF_MOVE · 288d3322
      Michael S. Tsirkin authored
      When memory is allocated on a wrong node, MPOL_MF_STRICT
      doesn't move it - it just fails the allocation.
      A simple way to reproduce the failure is with mlock=on
      realtime feature.
      
      The code comment actually says: "ensure policy won't be ignored"
      so setting MPOL_MF_MOVE seems like a better way to do this.
      
      Cc: qemu-stable@nongnu.org
      Signed-off-by: 's avatarMichael S. Tsirkin <mst@redhat.com>
      
      288d3322
  15. 18 Aug, 2014 1 commit
  16. 19 Jun, 2014 5 commits
  17. 18 Jun, 2014 1 commit