Clozure CPU usage limit?

[Maxxum]

I have been using Clozure (ccl) on Linux (core 2 duo) and OS X (core 2 duo)

My observations are:
* On Linux, Clozure uses one core maximally and doest not really use the other one. (Occasionally, it switches cores, but it just uses one at a time)
* On OS X, both cores are used for about 50%.

Because Clozure lets the OS schedule threads, the above is reasonable.

But it seems to me that Clozure can use, at most, the CPU resource that is equivalent to one core. I know MCL has this restriction of CPU usage on multi-CPU machines. I am wondering if Clozure has a similar restriction.

I am considering buying a Mac Pro. If there is this restriction, I would go for higher CPU frequency rather than more cores.

Thank you for any suggestion.

[Maxxum]

BTW, are the following about the same CPU power?
* ONE core from a 2.26GHz Quad-Core Xeon
* ONE core from a 2.26GHz Core 2 Duo

Thanks a lot.

[raffaelcavallaro]

The code below consistently uses about 140% cpu on a core2 duo machine (i.e., two cores). I have other code which does GUI drawing in addition to processor intensive things on other threads that regularly gets near 200% cpu on a dual core machine. IOW, CCL under Mac OS X 10.5 and 10.6 has no problem using all the available cpu resources, as long as you write code that does that.

Call this by doing (for example):
(cpu-maxer 10)

which will spawn 6 threads each running a slightly randomized takeuchi benchmark for 10 seconds, after which these 6 threads will be killed.

Code: Select all

(declaim (notinline tak))

(eval-when (:compile-toplevel :load-toplevel :execute)
  (defmacro with-elapsed-time (&body body)
  (let* ((startsym (gensym "STARTSYM")))
    `(let* ((,startsym (get-internal-real-time)))
       (flet ((elapsed-time-function ()
                (float (/ (- (get-internal-real-time) ,startsym)
                          internal-time-units-per-second) 0.0d0)))
         (symbol-macrolet ((elapsed-time (elapsed-time-function)))
                          (progn ,@body)))))))

(defun tak (x y z)
  (declare (double-float x y z)
           (ftype (function (double-float double-float double-float) double-float) tak)
           (optimize (speed 3 ) (safety 0) (debug 0) (compilation-speed 0) (space 0))
           #+lispworks(optimize (float 0) (fixnum-safety 0)))
  (the double-float
       (if (>= (the double-float y) (the double-float x))
         (the double-float z)
         (the double-float
              (tak
               (the double-float
                    (tak (the double-float (- (the double-float x) 1.0d0))
                         (the double-float y) (the double-float z)))
               (the double-float
                    (tak (the double-float (- (the double-float y) 1.0d0))
                         (the double-float z) (the double-float x)))
               (the double-float
                    (tak (the double-float (- (the double-float z) 1.0d0))
                         (the double-float x) (the double-float y))))))))

(defun cpu-maxer (seconds)
  (with-elapsed-time
      (dotimes (n 6)
        (process-run-function
         (format nil "thread number ~a" n)
         (lambda ()
           (loop
             until (> elapsed-time seconds)
             do
             (let* ((multiplier (+ 8 (random 2))))
               (tak (* 3.0d0 multiplier)
                    (* 2.0d0 multiplier)
                    (* 1.0d0 multiplier))))
           (process-kill ccl::*current-process*))))))


[Maxxum]

Thank you very much!

For each SINGLE thread, the maximum CPU power it can get is ONE core. Is this correct?

IOW, to use the entire CPU, there should be at least two threads?

[nuntius]

That is correct. No matter the language, a program must have at least 1 OS thread per core you want it to use in parallel.

Regarding frequency versus cores, some classic computer architecture papers prove that, all else equal, it is better to have one massive fast CPU than many smaller networked CPUs. However, CPU designers have hit a wall; so they've returned to multicore processors, and we may never go back.

[findinglisp]

BTW, are the following about the same CPU power?
* ONE core from a 2.26GHz Quad-Core Xeon
* ONE core from a 2.26GHz Core 2 Duo

Depends highly on the models of each because of cache size, microarchitecture, etc. Generally, a latest-gen Xeon family will be faster than a latest-gen Core 2 Duo family, but "Xeon" and "Core 2 Duo" cover a lot of ground as far as Intel processors go.

[S11001001]

When using other systems, note that you can't always take advantage of true parallelism on multiprocessor machines, even when OS-level threading is available.

While Clozure, SBCL, and probably most other Common Lisps supporting threads support true parallelism on compatible machines, it is not always the case for VMs in general. For example, CPython spawns OS-level threads, but protects the interpreter with a global interpreter lock, meaning that only one thread may execute Python code at a time. You can still have other threads waiting on I/O, and still have preemption on the Python level, which makes web servers like Zope practical, but you need to use multiple processes if you want true parallelism.

[findinglisp]

Ruby has a similar model to Python. There are multiple threads, but only one can run at a time. They are good for saving state in the call stack, but not for doing simultaneous processing on a SMP machine.