Text Size

Distributed Computing

The Kraken: affinity wrapper for a3 and a5 FahCores

Folding @ Home, and Other Distributed Computing Topics

The Kraken: affinity wrapper for a3 and a5 FahCores

Postby tear » Tue Mar 15, 2011 5:30 am

Hey, I've been working on FAH affinity wrapper over past several days.

Motivation: kernel scheduling sucks on 4p (ck scheduler didn't help either).

How it works: it wraps around FahCore binaries and sets CPU affinity as soon
as subsequent worker threads get created (timing is critical here, I'll
elaborate a bit later, looking for testing resources now :mrgreen: )

Target audience: 4p and/or anyone who's into pushing the envelope a little bit
further :) (can install gcc, run several commands, etc.)

Here's beta P2686 (same number of atoms as P6901) running below 6m35s
Code: Select all
[03:20:57] Project: 2686 (Run 9, Clone 9, Gen 95)
[03:20:57]
[03:20:57] Assembly optimizations on if available.
[03:20:57] Entering M.D.
[03:21:05] Mapping NT from 48 to 48
[03:21:07] Completed 0 out of 250000 steps  (0%)
[03:27:41] Completed 2500 out of 250000 steps  (1%)
[03:34:11] Completed 5000 out of 250000 steps  (2%)
[03:40:44] Completed 7500 out of 250000 steps  (3%)
[03:47:14] Completed 10000 out of 250000 steps  (4%)
[03:53:46] Completed 12500 out of 250000 steps  (5%)
[04:00:16] Completed 15000 out of 250000 steps  (6%)
[04:06:48] Completed 17500 out of 250000 steps  (7%)
[04:13:22] Completed 20000 out of 250000 steps  (8%)
[04:19:52] Completed 22500 out of 250000 steps  (9%)
[04:26:25] Completed 25000 out of 250000 steps  (10%)


I hope to be done with preliminary tests by end of the day tomorrow (Mountain Time).
Last edited by tear on Wed Mar 16, 2011 12:31 am, edited 1 time in total.
User avatar
tear
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 461
Joined: Sun Sep 12, 2010 3:27 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby tear » Wed Mar 16, 2011 12:31 am

Updated: Apr 12, 2013 [long overdue]

The Kraken 0.7-pre15 is available.

It's a prerelease but is mature enough to be shared.

Thanks to extensive research and work of firedfly of [H],
The Kraken now features novel approach to making FahCores
engage DLB (Dynamic Load Balancing).

Instead of restarting the FahCore, it now creates synthetic
load on every other CPU; said load "comes and goes" (with
certain duty cycle) until DLB is engaged or until deadline
is reached (5 minutes by default).

Such approach is much more effective at making FahCores
enable DLB than autorestarting. 100% success rates have
been observed.

All users are recommended to upgrade.

NOTE: on lower-thread-count machines (24 or less) with non-BA units,
      synthetic load may not succeed in triggering DLB; consequently
      first 5 minutes of folding may suffer from hindered performance

Pull from: http://darkswarm.org/thekraken-0.7-pre15.tar.gz

All users are encouraged to get acquainted with README.txt,
included below for your convenience. Changelog follows.

Code: Select all
$ cat README.txt

Contents:
0. License
1. What does The Kraken do?
2. How is The Kraken useful?
3. Supported configurations (PLEASE READ!)
4. Upgrade recommendations
4.1. Upgrade recommendations: from The Kraken 0.2
4.2. Upgrade recommendations: from The Kraken 0.3
4.3. Upgrading from The Kraken 0.4, The Kraken 0.6-pre4 or The Kraken 0.6
5. Building and installing The Kraken
5.1. Pre-requisites
5.2. Building and installation
6. Wrapping
6.1. Wrapping: V6 client
6.2. Wrapping: V7 client
6.3. Dynamic Load Balancing
7. Unwrapping
8. How do I know it's working?
9. Known issues and caveats
10. Support
11. Credits



0. License

  The Kraken is licensed under GNU General Public License version 2.
  See COPYING.txt for details.



1. What does The Kraken do?

  The Kraken wraps around FahCore binaries and sets CPU affinity as soon
  as subsequent worker threads get created.

  It also creates synthetic load with the intent of triggering DLB (Dynamic
  Load Balancing) which improves performance in majority of configurations.



2. How is The Kraken useful?

  Linux scheduler(s) tend to make sub-optimal decisions as far as FahCores
  are concerned. Multinode NUMA machines are affected most; local worker
  thread's memory may get allocated (paged in) on one node but then same
  worker thread usually gets migrated to a processor on another node thus
  defeating the concept of local memory.

  The Kraken sets CPU affinity at earliest possible moment thus ensuring
  that worker threads always use local memory (same-node page-ins).



3. Supported configurations

  The Kraken is supported on modern x86-64 Linux distributions running
  Folding@Home clients 6.34 and 7.

  The Kraken supports FahCores A3 and A5. Support for other FahCores
  may be added in future, as required.

  The Kraken supports single SMP client. Multi-SMP-client configurations
  are not supported at this time.



4. Upgrade recommendations

4.1. Upgrade recommendations: from The Kraken 0.2

    It is recommended to unwrap FahCores and remove (or archive) The Kraken
    from the system for the sake of housekeeping.

    1. Go to client directory (shutting the client down is not required)
    2. Run './thekraken-0.2 -u'; it should give output similar to the following:

    $ ./thekraken-0.2 -u
    thekraken: The Kraken 0.2
    thekraken: Processor affinity wrapper for Folding@Home
    thekraken: The Kraken comes with ABSOLUTELY NO WARRANTY; licensed under GPLv2
    thekraken: performing uninstallation
    thekraken: FahCore_a5.exe: wrapper succesfully uninstalled
    thekraken: FahCore_a3.exe: wrapper succesfully uninstalled
    thekraken: finished uninstallation
    $

    3. Run 'rm -f thekraken-0.2*'
    4. Follow instructions in 6.1 or 6.2, depending on configuration.



4.2. Upgrade recommendations: from The Kraken 0.3

    It is recommended to unwrap FahCores and remove (or archive) The Kraken
    from the system for the sake of housekeeping.

    If you downloaded and installed The Kraken 0.3 (which wasn't very
    fortunate release), there are two recommended action items --
    unwrapping FahCores in client directory and removing The Kraken
    from /usr/local/bin.

    1. Run (as root) '/usr/local/bin/thekraken -yu /var/lib/fahclient';
       on Ubuntu prepend the line with 'sudo'.
    2. Run (as root) 'rm -f /usr/local/bin/thekraken'; on Ubuntu
       prepend the line with 'sudo'.
    3. Follow instructions in 6.1 or 6.2, depending on configuration.



4.3. Upgrading from The Kraken 0.4, The Kraken 0.6-pre4 or The Kraken 0.6

    1. Build and install The Kraken per section 5.
    2. Go to client directory (shutting the client down is not required)
    3. Run 'thekraken -u'; it should give output similar to the following:


       $ thekraken -u
       thekraken: The Kraken 0.7-pre11 (compiled Sun May 20 19:36:47 MDT 2012 by fah@tentacle)
       thekraken: Processor affinity wrapper for Folding@Home
       thekraken: The Kraken comes with ABSOLUTELY NO WARRANTY; licensed under GPLv2
       thekraken: performing uninstallation from .
       thekraken: /home/fah/fah-6.34/FahCore_a3.exe: wrapper succesfully uninstalled
       thekraken: /home/fah/fah-6.34/FahCore_a5.exe: wrapper succesfully uninstalled
       thekraken: removing configuration file
       thekraken: finished uninstallation, 2 out of 2 file(s) processed
       $


    4. Run 'thekraken -w'; it should give output similar to the following:

       $ thekraken -i
       thekraken: The Kraken 0.7-pre11 (compiled Sun May 20 19:36:47 MDT 2012 by fah@tentacle)
       thekraken: Processor affinity wrapper for Folding@Home
       thekraken: The Kraken comes with ABSOLUTELY NO WARRANTY; licensed under GPLv2
       thekraken: performing installation to .
       thekraken: /home/fah/fah-6.34/FahCore_a3.exe: wrapper succesfully installed
       thekraken: /home/fah/fah-6.34/FahCore_a5.exe: wrapper succesfully installed
       thekraken: finished installation, 2 out of 2 files processed
       $



5. Building and installing The Kraken

5.1. Pre-requisites

    1. Make sure that (where appropriate)
       - NUMA is enabled,
       - node interleave is disabled,
       - ACPI SRAT is enabled

    2. Make sure that make and gcc are installed; on Ubuntu issue

       sudo apt-get install gcc make

    3. Make sure that all target FahCores have been downloaded
       by the client.



5.2. Building and installation

    1. Run 'make'
    2. Run 'make install' as root (on Ubuntu, run 'sudo make install')
   
    Voila. The Kraken is available in /usr/bin.



6. Wrapping

6.1. Wrapping: V6 client

    1. Stop the client
    2. Go to client directory
    3. Run 'thekraken -w'

       Doing so should result in something along the following:

       $ thekraken -i
       thekraken: The Kraken 0.7-pre11 (compiled Sun May 20 19:36:47 MDT 2012 by fah@tentacle)
       thekraken: Processor affinity wrapper for Folding@Home
       thekraken: The Kraken comes with ABSOLUTELY NO WARRANTY; licensed under GPLv2
       thekraken: performing installation to .
       thekraken: /home/fah/fah-6.34/FahCore_a3.exe: wrapper succesfully installed
       thekraken: /home/fah/fah-6.34/FahCore_a5.exe: wrapper succesfully installed
       thekraken: finished installation, 2 out of 2 files processed
       $

    4. Re-start the client.



6.2. Wrapping: V7 client

    As single V7 client installation may be run off more than one "home"
    directory, wrapping process is little more complicated.
    It is imperative client's home directory is determined prior to
    wrapping.

    1. Make sure the client is running (and folding)
    2. Determine client's "home" directory; run (as root; on Ubuntu prepend
       with 'sudo'):

       stat /proc/$(ps -eocomm=,pid= | awk '/^FahCo[^[]/ { print $NF }')/exe | head -1 | cut -f 3 -d \` | sed s=cores.*\$=cores/=

       It should return path, such as: '/var/lib/fahclient/cores/'.
       Make note of this path.

    3. Pause the client/slot
    4. Run (as root; on Ubuntu prepend with 'sudo'):

       thekraken -w path-determined-in-step-2

       Doing so should result in something along the following:

       $ thekraken -i /var/lib/fahclient/cores/
       thekraken: The Kraken 0.4 (compiled Tue Apr 12 20:27:47 MDT 2011 by fah@tentacle)
       thekraken: Processor affinity wrapper for Folding@Home
       thekraken: The Kraken comes with ABSOLUTELY NO WARRANTY; licensed under GPLv2
       thekraken: performing installation to /var/lib/fahclient/cores/
       thekraken: descend into /var/lib/fahclient/cores/www.stanford.edu and all other subdirectories [Y/n]?

       If path appears correct, confirm with 'y' and press Enter. This should result
       in something along the following:

       thekraken: /var/lib/fahclient/cores/www.stanford.edu/~pande/Linux/AMD64/Core_a3.fah/FahCore_a3: wrapper succesfully installed
       thekraken: /var/lib/fahclient/cores/www.stanford.edu/~pande/Linux/AMD64/Core_a5.fah/FahCore_a5: wrapper succesfully installed
       thekraken: finished installation, 2 out of 2 files processed
       $

    5. Resume the client/slot



6.3. Dynamic Load Balancing

    Background: GROMACS employs Dynamic Load Balancing (DLB)
    feature that aims at improving performance.

    GROMACS configuration used by FahCores enables DLB the moment
    cumulative performance loss due to load imbalance exceeds 5%.

    When enabled, DLB reduces times of bigadv units by noticable
    amount of time. Reports include reduction of 30s with P6903
    and 45 seconds with P6904 (sometimes more).

    Depending on WU and system configuration (or even system state),
    DLB gets enabled in a way that may appear random (sometimes it's
    several minutes into WU; at other times it may be as late
    as 90% into WU, sometimes it doesn't engage at all).

    The Kraken 0.7 features a novel way of making FahCores enable
    DLB. It creates synthetic load on every other CPU; said load comes
    and goes (with certain duty cycle) until DLB is engaged or until
    deadline is reached (5 minutes by default).

    DLB triggering is enabled by default. To disable it, add '-c dlbload=0'
    parameter to the command line, when wrapping, e.g.
    'thekraken -w -c dlbload=0'.
    If already wrapped: unwrap, then re-wrap with '-c dlbload=0'.
    Stopping the client is not required.



7. Unwrapping

    Follow wrapping instructions but replace 'thekraken -w' with 'thekraken -u'.



8. How do I know it's working?

    Method #1:

        While folding, run 'top'. Look for process consuming most CPU. 'COMMAND'
        of that process should read 'thekraken-FahCo', e.g.:

           PID  USER     PR  NI  VIRT  RES  SHR S   %CPU %MEM    TIME+   P COMMAND
          8411  fah      39  19 6452m 4.5g 3188 S 4801.6 14.3  52424:56 24 thekraken-FahCo


    Method #2 (examination of FahCores' affinities):

        While folding, run:
   
          for i in $(ps -L auxw | awk '/FahCo[^[]/ { if ($4 > 10) print $3 }' | sort -n -k1,1) ; do taskset -pc $i  ; done

        Doing so should give you sequential processor numbers starting with 0 in its
        output, e.g.:

        pid 41666's current affinity list: 0
        pid 41669's current affinity list: 1
        pid 41670's current affinity list: 2
        pid 41671's current affinity list: 3
        pid 41672's current affinity list: 4
        pid 41673's current affinity list: 5
        pid 41674's current affinity list: 6
        pid 41675's current affinity list: 7
        pid 41676's current affinity list: 8
        pid 41677's current affinity list: 9
        pid 41678's current affinity list: 10
        pid 41679's current affinity list: 11
        pid 41680's current affinity list: 12
        pid 41681's current affinity list: 13
        pid 41682's current affinity list: 14
        pid 41683's current affinity list: 15
        pid 41684's current affinity list: 16
        pid 41685's current affinity list: 17
        pid 41686's current affinity list: 18
        pid 41687's current affinity list: 19
        pid 41688's current affinity list: 20
        pid 41689's current affinity list: 21
        pid 41690's current affinity list: 22
        pid 41691's current affinity list: 23
        pid 41692's current affinity list: 24
        pid 41693's current affinity list: 25
        pid 41694's current affinity list: 26
        pid 41695's current affinity list: 27
        pid 41696's current affinity list: 28
        pid 41697's current affinity list: 29
        pid 41698's current affinity list: 30
        pid 41699's current affinity list: 31
        pid 41700's current affinity list: 32
        pid 41701's current affinity list: 33
        pid 41702's current affinity list: 34
        pid 41703's current affinity list: 35
        pid 41704's current affinity list: 36
        pid 41705's current affinity list: 37
        pid 41706's current affinity list: 38
        pid 41707's current affinity list: 39
        pid 41708's current affinity list: 40
        pid 41709's current affinity list: 41
        pid 41710's current affinity list: 42
        pid 41711's current affinity list: 43
        pid 41712's current affinity list: 44
        pid 41713's current affinity list: 45
        pid 41714's current affinity list: 46
        pid 41715's current affinity list: 47

   
   
9. Know issues and caveats

    CAUTION: FahCore_a5 is known to be problematic at user-induced shutdowns*.
             To be on a safe side make a backup of complete client directory
             before hitting Ctrl+C!

    To tell whether checkpoint was written correctly check the size
    of work/wudata_XX.ckp file (XX being current slot number).
    It should be 75160 (for core A5). If it's not -- better switch to backed up
    directory.

    *) http://foldingforum.org/viewtopic.php?f=55&t=17774

   
    CAUTION: The Kraken may produce sub-optimal results
             when nodes with no memory installed are present.



10. Support

    Using AMDZone forum is recommended. Please post in this thread:

    http://www.amdzone.com/phpbb3/viewtopic.php?f=521&t=138463



11. Credits

    The Kraken was written and is maintained by Kris Rusocki <kszysiu@gmail.com>
    Synthload DLB research and coding was done by Stephen Gordon <firedfly@gmail.com>

    Special thanks go to:

        brutis at AMDZone -- excellent V7 test feedback

        sfield -- original DLB enagement observation

        musky at [H]ardForum -- alternative method for installation
                                confirmation

        firedfly at [H]ardForum -- help with troubleshooting
                                   client-on-remote-filesystem-with-off-clock
                                   issue



2012-06-15 -- Prerelease (thekraken-0.7-pre15)
-- convert startup deadline to only warn the user
  of bad WU instead of killing FahCore
-- make sure that dlbload gets started only once
  (could manifest with <5 min./frame WUs)


2012-06-05 -- Prerelease (thekraken-0.7-pre14)
-- support FahCore_a4
-- V7: unblock appropriate signals so dlbload
  can be stopped properly


2012-06-03 -- Prerelease (thekraken-0.7-pre13)
-- detect logfile fd with greater deal of flexibility
  to accommodate V7 client


2012-06-02 -- Prerelease (thekraken-0.7-pre12)
-- add startup deadline so broken units fail early
-- include improved logging infrastructure
-- FahCore installation process is now called 'wrapping'
  with dedicated command line option: -w; -i is preserved
  for backward compatibility


2012-05-20 -- Prerelease (thekraken-0.7-pre11)
-- resolves build issues in certain configurations


2012-05-20 -- Prerelease (thekraken-0.7-pre10)
-- use dynamic log fd detection w/DLB load


2012-05-20 -- Prerelease (thekraken-0.7-pre9)
-- incorporate novel approach to making FahCores engage
  Dynamic Load Balancing (Stephen Gordon)
-- autorestart DLB triggering method is no longer viable,
  drop it
-- 'np' override has been made obsolete by updates to V7
  client, drop it
-- prevent redundant SIGTERM from being delivered to FahCore
  if it's already shutting down due to SIGINT/Ctrl+C
-- play nicer with shell's job control
-- include more robust parsing of command line and config
  file variables
-- include startcpu configuration variable for advanced users
-- attempt to prevent root from running 'make install' initially


2011-12-04 -- New release (thekraken-0.6)
-- allow The Kraken to operate correctly on remote
  filesystems with inaccurate timekeeping
-- use stat() rather than lstat()
-- minor makefile update
-- documentation update


2011-07-30 -- Prerelease (thekraken-0.6-pre4)
-- The Kraken now sports autorestart feature (aims
  at engaging Dynamic Load Balancing)
-- avoid excessive prompts when operating recursively
-- don't use non-reentrant functions in signal handler
-- Makefile revamp
-- couple cosmetic documentation updates


2010-04-12 -- New release (thekraken-0.4)
-- huge documentation update
-- set default install directory to /usr/bin rather than
  /usr/local/bin which doesn't "work" with sudo
-- configuration file support
-- number-of-threads override feature for six-core machines
  running bigadv WUs (see section 6.3 of the README.txt file)
-- cosmetic code cleanups
-- Makefile: minor tweaks


2010-04-02 -- New release (thekraken-0.3)
-- support V7 client: traverse directory structure
-- support V7 client: preserve file ownership (if run by root)
-- option parsing code is now more civilized (getopt)
-- first release with half-decent documentation


2010-03-15 -- Initial release (thekraken-0.2)


Let know if you have any questions or comments.
Last edited by tear on Sat Apr 13, 2013 5:14 am, edited 6 times in total.
User avatar
tear
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 461
Joined: Sun Sep 12, 2010 3:27 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby WitchDoctorB » Wed Mar 16, 2011 2:26 am

Sweet! Now if only I wouldn't have had those margaritas at dinner I d be able to try it tonight, have to wait till the am. :D
Image Image
User avatar
WitchDoctorB
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 404
Joined: Thu Jul 10, 2008 1:14 pm
Location: The marsh, south of the swamps, in the eyes of Gustav and Isaac

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby BaBa » Wed Mar 16, 2011 5:42 am

Will give this a try after i get some sleep, many thanks :)
BaBa
 
Posts: 57
Joined: Mon Mar 14, 2011 5:18 am

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby BaBa » Thu Mar 17, 2011 2:19 am

OK got it working (helps if you spell kraken correctly :lol: )

Will post results after a few frames,thanks again to Tear :)
BaBa
 
Posts: 57
Joined: Mon Mar 14, 2011 5:18 am

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby tear » Thu Mar 17, 2011 2:47 am

Awesome! Standing by.

Did you have a chance to run the 'verification' line?
(the long 'for' line)
User avatar
tear
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 461
Joined: Sun Sep 12, 2010 3:27 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby BaBa » Thu Mar 17, 2011 3:20 am

Tear you are a genius! frame time average across 8 frames down to 6.12 from 6.48 :mrgreen:

Verification run and affinity seem to be set as per your post, if this continues then 6901 gives a theoretical PPD of 399K :!:
BaBa
 
Posts: 57
Joined: Mon Mar 14, 2011 5:18 am

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby tear » Thu Mar 17, 2011 3:23 am

Kickass! :mrgreen:

Still, please report any issues ;)
User avatar
tear
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 461
Joined: Sun Sep 12, 2010 3:27 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby BaBa » Thu Mar 17, 2011 3:36 am

Time for bed now will post some times tomorrow and look at the spread a little closer.

Thanks again :)
BaBa
 
Posts: 57
Joined: Mon Mar 14, 2011 5:18 am

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby BaBa » Thu Mar 17, 2011 1:53 pm

Looked at the frame times from the 6901 i completed this morning they are within 4 seconds ranging from 6.11 to 6.15 with average of 6.12 :D

They did jump quite sharply when i posted from the rig in question this morning to around 7 mins, this is not realy a problem for a 99% dedicated folding rig.

Picked up a 2685 after the 6901, frame times for this are 6.30 but its the first non 6901 ive picked up on linux so dont have anything to compare it to :?:
BaBa
 
Posts: 57
Joined: Mon Mar 14, 2011 5:18 am

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby tear » Thu Mar 17, 2011 3:11 pm

Of big P26xx, only P2686 runs as fast as P690x.
Others run tad slower, with notable mention of
P2684 which runs much, much slower (9m40s here
:shock: ).

With worker threads bound to all CPUs frame times
will be affected by computer use, it's the nature
of the beast...
User avatar
tear
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 461
Joined: Sun Sep 12, 2010 3:27 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby BaBa » Thu Mar 17, 2011 11:24 pm

Yes 2684's always have been a pain.

Hope the ES chips turn out to be overclockable even if
they only reach 2.3 i would be happy,actually even at
2.1 i'm happy :lol:
BaBa
 
Posts: 57
Joined: Mon Mar 14, 2011 5:18 am

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby 10e » Mon Mar 21, 2011 12:43 am

I had my ES chips at 2.6Ghz running 100% stable for weeks prior to purchasing my SuperMicro H8QGI-F motherboard.

They were able to hit 2.8Ghz without issue and still run below 62 celsius at full bore with only air coolers on them.

These chips have monstrous potential.

Back on topic and apologies for the digression:

With Kraken I am seeing 7:21 times-per-frame consistently on P6901s with my 1.9Ghz Opteron 6168s. I am very happy with this, as it is basically what I should theoretically be getting.

Thanks very much Tear for your contribution(s).

I'll try and find the ES thread later. I may still have the R-W scripts I used to write MSRs for the ES chips that allowed both multiplier and VID/VCORE changes.

Cheers!
10e
 
Posts: 1
Joined: Fri Mar 18, 2011 6:56 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby brutis » Mon Mar 21, 2011 12:48 am

10e wrote:I had my ES chips at 2.6Ghz running 100% stable for weeks prior to purchasing my SuperMicro H8QGI-F motherboard.

They were able to hit 2.8Ghz without issue and still run below 62 celsius at full bore with only air coolers on them.

These chips have monstrous potential.

Makes me want to put mine on water even more now!

And Welcome to the AMDZone!!!
Image
Image
User avatar
brutis
K8 Opteron (SledgeHammer) Moderator
K8 Opteron (SledgeHammer) Moderator
 
Posts: 6045
Joined: Sat Mar 06, 2004 2:36 pm
Location: so there I was...

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby Jmtyra » Thu Mar 24, 2011 3:53 am

Thanks for putting this out tear! I didn't have much luck this initially, as it dropped my TPF into the hours range. It might have been something else, but after clearing the Kraken I was back to ~10min range on TPF.

When I get SR-2 #3 going, I'll try it again to see if I can replicate the issue. Hopefully I'll remember to grab the log this time if the issue replicates itself.

I'm sure it's something stupid that I did, but if it's not I'll try to give feedback to help out development.

Chat with you later man :)

-JT
Jmtyra
 
Posts: 3
Joined: Wed Mar 23, 2011 11:37 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby tear » Thu Mar 24, 2011 10:27 am

Hey, thanks for the feedback. Always appreciated.

Hours!? No, that doesn't sound right at all.

Whenever you feel like experimenting again, please start the client
(with thekraken, ofc) and (while client is running and folding)
do the following and provide the output please:

Code: Select all
for i in $(ps -L auxw | grep [^]]FahCo  | awk '{ if ($4 > 10) print $3 }' | sort -n -k1,1) ; do taskset -pc $i  ; done

Code: Select all
for i in $(ps -L auxw | grep [^]]FahCo  | awk '{ print $3 }' | sort -n -k1,1) ; do taskset -pc $i  ; done

Code: Select all
ps -L auxw | awk '{ if ($4 > 10) print }'

Code: Select all
ls -l /sys/devices/system/node/


Also, paste contents of thekraken.log.

What operating system are you running?
Are you using BFS scheduler?
Did you go over step 1 checklist? (NUMA, et al.)

Thanks,
tear
User avatar
tear
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 461
Joined: Sun Sep 12, 2010 3:27 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby Pietro sk » Thu Mar 24, 2011 1:02 pm

brutis wrote:
10e wrote:I had my ES chips at 2.6Ghz running 100% stable for weeks prior to purchasing my SuperMicro H8QGI-F motherboard.

They were able to hit 2.8Ghz without issue and still run below 62 celsius at full bore with only air coolers on them.

These chips have monstrous potential.

Makes me want to put mine on water even more now!

And Welcome to the AMDZone!!!

OHNOES !
WATER-MAYBE 3GIGS POSSIBLE :!:
G34 BLACK EDITION - KABOOM !!!!!!!
Image
The famous intel lawsuit Image Moron is playing VIDEO-game
Not biased Cinebench ? Think again ... They say it´s "finetuned" for intel CPU´s .. Image
User avatar
Pietro sk
K8 Athlon 64 X2 (Windsor) Elite Boarder
K8 Athlon 64 X2 (Windsor) Elite Boarder
 
Posts: 3932
Joined: Fri Jan 08, 2010 3:59 pm
Location: Le sarcasm..

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby Jmtyra » Thu Mar 24, 2011 7:04 pm

I'm working on SR-2 #3 today, and will try to replicate the issue. It very well may have been user error on my part! :D

tear wrote:What operating system are you running?
Are you using BFS scheduler?
Did you go over step 1 checklist? (NUMA, et al.)

--> Using Ubuntu Desktop 10.10 x64
--> Not 100% certain. Is that the "sudo apt-get install gcc" bit, or is there something else I need to do?
--> Yup! Running NUMA enabled on all of the SR-2 boards now. :wink:

Edit: Did a bit o' googling on BFS (found here) and it looks like I actually *haven't* installed that. Whoops! :( Can we add that to the instructions, or am I going the wrong direction?

Thanks again tear!
-JT
Jmtyra
 
Posts: 3
Joined: Wed Mar 23, 2011 11:37 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby tear » Fri Mar 25, 2011 12:28 am

Hey JT,


I would actually prefer you do not install BFS at this time.

Other than that, I think you're looking good.

Whenever feasible please try reproducing the issue and gather
requested data so we can possibly see what went wrong :-)

Let know if you have further questions or doubts.

Thanks,
tear
User avatar
tear
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 461
Joined: Sun Sep 12, 2010 3:27 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby MIBW » Tue Mar 29, 2011 5:15 am

Hey Tear - you legend, many thanks for this tool.

SR2s - Ubuntu 10.10, BFS - 28% better PPD than windows - 8% of that due to your tool...

What are your thoughts on how to install with new V7 client?
MIBW
 
Posts: 4
Joined: Tue Mar 29, 2011 4:50 am

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby brutis » Tue Mar 29, 2011 10:36 am

Welcome to the AMDZone MIBW!
Image
Image
User avatar
brutis
K8 Opteron (SledgeHammer) Moderator
K8 Opteron (SledgeHammer) Moderator
 
Posts: 6045
Joined: Sat Mar 06, 2004 2:36 pm
Location: so there I was...

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby tear » Tue Mar 29, 2011 2:14 pm

You're welcome, MIBW.

BTW, on future builds you can skip the BFS and go straight to The Kraken.

To answer your question --

Principle is the same, yet, currently, The Kraken doesn't
process cores with V7 names (no ".exe" suffix, i.e. FahCore_a3
vs. traditional FahCore_a3.exe).

There are other caveats involved too:
(a) FahCores are located in a directory structure (/var/lib/fahclient/cores)
   rather than single directory,
(b) all client files are owned by a dedicated user (fahclient) thus
   extra care must be taken when making modifications to any client
   files (including FahCores renaming/substitution)

That said, The Kraken can't be used OOTB with V7. Of course, one could try
and work around that with kludges here and there but that would be going
beyond ordinary use.

Do you see a need for V7 support?


Thanks,
tear
User avatar
tear
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 461
Joined: Sun Sep 12, 2010 3:27 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby Tobit » Tue Mar 29, 2011 4:11 pm

tear wrote:Do you see a need for V7 support?

YES, please. There are quite a few of us on Team 33 alone now running it and I know some of us want to play with v7. If you can add v7 support without if having it take up too much of your time, I am sure it will be greatly appreciated.
Tobit
 
Posts: 7
Joined: Tue Mar 29, 2011 4:08 pm

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby MIBW » Wed Mar 30, 2011 5:11 am

tear wrote:Do you see a need for V7 support?


Oh hells yeah. Wish I could say there was big money in it for you, but the lack of Krakken is the main reason I am holding off on V7.
MIBW
 
Posts: 4
Joined: Tue Mar 29, 2011 4:50 am

Re: The Kraken: affinity wrapper for a3 and a5 FahCores

Postby tear » Wed Mar 30, 2011 10:43 am

Noted. I should have something ready by next week.

Welcome to the Zone!
User avatar
tear
K7 Athlon XP (Palomino) Junior Boarder
K7 Athlon XP (Palomino) Junior Boarder
 
Posts: 461
Joined: Sun Sep 12, 2010 3:27 pm

Next

Return to Distributed Computing

Who is online

Users browsing this forum: Bing [Bot] and 1 guest

cron