Clozure CPU usage limit?

Discussion of Common Lisp
Post Reply
Maxxum
Posts: 6
Joined: Wed Jul 01, 2009 8:28 pm

Clozure CPU usage limit?

Post by Maxxum » Mon Sep 14, 2009 1:20 pm

I have been using Clozure (ccl) on Linux (core 2 duo) and OS X (core 2 duo)

My observations are:
* On Linux, Clozure uses one core maximally and doest not really use the other one. (Occasionally, it switches cores, but it just uses one at a time)
* On OS X, both cores are used for about 50%.

Because Clozure lets the OS schedule threads, the above is reasonable.

But it seems to me that Clozure can use, at most, the CPU resource that is equivalent to one core. I know MCL has this restriction of CPU usage on multi-CPU machines. I am wondering if Clozure has a similar restriction.

I am considering buying a Mac Pro. If there is this restriction, I would go for higher CPU frequency rather than more cores.

Thank you for any suggestion.

Maxxum
Posts: 6
Joined: Wed Jul 01, 2009 8:28 pm

Re: Clozure CPU usage limit?

Post by Maxxum » Mon Sep 14, 2009 1:53 pm

BTW, are the following about the same CPU power?
* ONE core from a 2.26GHz Quad-Core Xeon
* ONE core from a 2.26GHz Core 2 Duo

Thanks a lot.

raffaelcavallaro
Posts: 5
Joined: Mon Feb 02, 2009 3:00 pm

Re: Clozure CPU usage limit?

Post by raffaelcavallaro » Mon Sep 14, 2009 2:17 pm

The code below consistently uses about 140% cpu on a core2 duo machine (i.e., two cores). I have other code which does GUI drawing in addition to processor intensive things on other threads that regularly gets near 200% cpu on a dual core machine. IOW, CCL under Mac OS X 10.5 and 10.6 has no problem using all the available cpu resources, as long as you write code that does that.

Call this by doing (for example):
(cpu-maxer 10)

which will spawn 6 threads each running a slightly randomized takeuchi benchmark for 10 seconds, after which these 6 threads will be killed.

Code: Select all

(declaim (notinline tak))

(eval-when (:compile-toplevel :load-toplevel :execute)
  (defmacro with-elapsed-time (&body body)
  (let* ((startsym (gensym "STARTSYM")))
    `(let* ((,startsym (get-internal-real-time)))
       (flet ((elapsed-time-function ()
                (float (/ (- (get-internal-real-time) ,startsym)
                          internal-time-units-per-second) 0.0d0)))
         (symbol-macrolet ((elapsed-time (elapsed-time-function)))
                          (progn ,@body)))))))

(defun tak (x y z)
  (declare (double-float x y z)
           (ftype (function (double-float double-float double-float) double-float) tak)
           (optimize (speed 3 ) (safety 0) (debug 0) (compilation-speed 0) (space 0))
           #+lispworks(optimize (float 0) (fixnum-safety 0)))
  (the double-float
       (if (>= (the double-float y) (the double-float x))
         (the double-float z)
         (the double-float
              (tak
               (the double-float
                    (tak (the double-float (- (the double-float x) 1.0d0))
                         (the double-float y) (the double-float z)))
               (the double-float
                    (tak (the double-float (- (the double-float y) 1.0d0))
                         (the double-float z) (the double-float x)))
               (the double-float
                    (tak (the double-float (- (the double-float z) 1.0d0))
                         (the double-float x) (the double-float y))))))))

(defun cpu-maxer (seconds)
  (with-elapsed-time
      (dotimes (n 6)
        (process-run-function
         (format nil "thread number ~a" n)
         (lambda ()
           (loop
             until (> elapsed-time seconds)
             do
             (let* ((multiplier (+ 8 (random 2))))
               (tak (* 3.0d0 multiplier)
                    (* 2.0d0 multiplier)
                    (* 1.0d0 multiplier))))
           (process-kill ccl::*current-process*))))))

Maxxum
Posts: 6
Joined: Wed Jul 01, 2009 8:28 pm

Re: Clozure CPU usage limit?

Post by Maxxum » Mon Sep 14, 2009 3:13 pm

Thank you very much!

For each SINGLE thread, the maximum CPU power it can get is ONE core. Is this correct?

IOW, to use the entire CPU, there should be at least two threads?

nuntius
Posts: 538
Joined: Sat Aug 09, 2008 10:44 am
Location: Newton, MA

Re: Clozure CPU usage limit?

Post by nuntius » Mon Sep 14, 2009 8:15 pm

That is correct. No matter the language, a program must have at least 1 OS thread per core you want it to use in parallel.

Regarding frequency versus cores, some classic computer architecture papers prove that, all else equal, it is better to have one massive fast CPU than many smaller networked CPUs. However, CPU designers have hit a wall; so they've returned to multicore processors, and we may never go back.

findinglisp
Posts: 447
Joined: Sat Jun 28, 2008 7:49 am
Location: Austin, TX
Contact:

Re: Clozure CPU usage limit?

Post by findinglisp » Sun Sep 20, 2009 1:32 am

Maxxum wrote:BTW, are the following about the same CPU power?
* ONE core from a 2.26GHz Quad-Core Xeon
* ONE core from a 2.26GHz Core 2 Duo
Depends highly on the models of each because of cache size, microarchitecture, etc. Generally, a latest-gen Xeon family will be faster than a latest-gen Core 2 Duo family, but "Xeon" and "Core 2 Duo" cover a lot of ground as far as Intel processors go.
Cheers, Dave
Slowly but surely the world is finding Lisp. http://www.findinglisp.com/blog/

S11001001
Posts: 5
Joined: Tue Sep 30, 2008 9:18 pm
Location: Indiana, USA
Contact:

Re: Clozure CPU usage limit?

Post by S11001001 » Sun Sep 20, 2009 2:13 pm

When using other systems, note that you can't always take advantage of true parallelism on multiprocessor machines, even when OS-level threading is available.

While Clozure, SBCL, and probably most other Common Lisps supporting threads support true parallelism on compatible machines, it is not always the case for VMs in general. For example, CPython spawns OS-level threads, but protects the interpreter with a global interpreter lock, meaning that only one thread may execute Python code at a time. You can still have other threads waiting on I/O, and still have preemption on the Python level, which makes web servers like Zope practical, but you need to use multiple processes if you want true parallelism.

findinglisp
Posts: 447
Joined: Sat Jun 28, 2008 7:49 am
Location: Austin, TX
Contact:

Re: Clozure CPU usage limit?

Post by findinglisp » Mon Sep 21, 2009 7:48 am

Ruby has a similar model to Python. There are multiple threads, but only one can run at a time. They are good for saving state in the call stack, but not for doing simultaneous processing on a SMP machine.
Cheers, Dave
Slowly but surely the world is finding Lisp. http://www.findinglisp.com/blog/

Post Reply