Hello,

I beleive the .travis.yml does work but it is still under active development.

The Ubuntu instructions in the linux developers page have been tested on a clean install and should work (https://surfer.nmr.mgh.harvard.edu/fswiki/freesurfer_linux_developers_page).

However I have not yet tested a build on a Debian system. I will make an attempt to build today on a clean debian "jessie" install. But I have one question, what branch are you attempting to build, because their is no "stable5_branch-5622-gf3d0bcd" branch in the git repo?

-Zeke

On 02/18/2016 09:59 AM, Yaroslav Halchenko wrote:

Hi Z K et al,

What would be the recommended/tested debian/ubuntu release to approach the build with some guarantee of success? Is the .travis.yml "exercised" (failed to quickly find a sign of it on travis-ci.org)?

I have tried on debian jessie to build stable5_branch-5622-gf3d0bcd of the git/annex repo, using following command:

 { ./setup_configure && ./configure --prefix=/usr/local/freesurfer/dev --with-pkgs-dir=${PWD}/../freesurfer-packages/centos6-x86_64-packages --disable-Werror --with-petsc-dir=/usr/lib/x86_64-linux-gnu && make -j3; } 2>&1 | tee ../freesurfer-upstream.buildlog1.txt

and before that installing various build depends and helpers through

 apt-get install libtool automake gfortran libglu1-mesa-dev libfreetype6-dev uuid-dev libxmu-dev libxmu-headers libxi-dev libx11-dev libxt-dev libxaw7-dev liblapack-dev tcsh curl git libmpich-dev git git-annex petsc-dev make

but the build fails with

mv -f .deps/xmlToHtml.Tpo .deps/xmlToHtml.Po /bin/bash ../libtool --tag=CC --mode=link g++ -L/usr/lib64 -L/usr/X11R6/lib64 -fopenmp -L/home/yoh/deb/gits/pkg-exppsy/freesurfer-upstream/../freesurfer-packages/centos6-x86_64-packages/mni/current/lib -L/home/yoh/deb/gits/pkg-exppsy/freesurfer-upstream/../freesurfer-packages/centos6-x86_64-packages/vxl/current/lib -L/home/yoh/deb/gits/pkg-exppsy/freesurfer-upstream/../freesurfer-packages/centos6-x86_64-packages/itk/current/lib/InsightToolkit -o xmlToHtml xmlToHtml.o ../xml2/libxml2.a -lz mv -f .deps/fsPrintHelp-fsPrintHelp.Tpo .deps/fsPrintHelp-fsPrintHelp.Po ../libtool: line 469: CDPATH: command not found /bin/bash ../libtool --tag=CC --mode=link g++ -DBUILD_MAIN -L/usr/lib64 -L/usr/X11R6/lib64 -fopenmp -L/home/yoh/deb/gits/pkg-exppsy/freesurfer-upstream/../freesurfer-packages/centos6-x86_64-packages/mni/current/lib -L/home/yoh/deb/gits/pkg-exppsy/freesurfer-upstream/../freesurfer-packages/centos6-x86_64-packages/vxl/current/lib -L/home/yoh/deb/gits/pkg-exppsy/freesurfer-upstream/../freesurfer-packages/centos6-x86_64-packages/itk/current/lib/InsightToolkit -o fsPrintHelp fsPrintHelp-fsPrintHelp.o ../xml2/libxml2.a -lz ../libtool: line 469: CDPATH: command not found libtool: Version mismatch error. This is libtool 2.4.2 Debian-2.4.2-1.11, but the libtool: definition of this LT_INIT comes from an older release. libtool: You should recreate aclocal.m4 with macros from libtool 2.4.2 Debian-2.4.2-1.11 libtool: and run autoconf again. Makefile:1674: recipe for target 'xmlToHtml' failed make[3]: *** [xmlToHtml] Error 63 make[3]: *** Waiting for unfinished jobs.... libtool: Version mismatch error. This is libtool 2.4.2 Debian-2.4.2-1.11, but the libtool: definition of this LT_INIT comes from an older release. libtool: You should recreate aclocal.m4 with macros from libtool 2.4.2 Debian-2.4.2-1.11 libtool: and run autoconf again. Makefile:1658: recipe for target 'fsPrintHelp' failed make[3]: *** [fsPrintHelp] Error 63

suggesting that not full auto/libtool'ification was carried out by ./setup_configure, but I didn't check detail yet.

full log available at http://www.onerussian.com/tmp/freesurfer-upstream.buildlog1.txt

Thanks in advance for the hints

On Tue, 02 Feb 2016, Z K wrote:

...

Hello Alle,

...

The naming of "configure.in" vs. "configure.ac" is not the issue. The script is erroring out because file 'm4/autotroll.m4' does not exist. Do you have that file (it should have been included in the checkout)?

...

I would recommend using our git repo which mirrors our CVS repo. Please see the following page:

...

https://surfer.nmr.mgh.harvard.edu/fswiki/freesurfer_linux_developers_page

...

Hope this helps,

...

-Zeke

...

On 02/02/2016 04:11 AM, Alle Meije Wink wrote:

...

I'm trying to build FreeSurfer from source on my Debian 8 (jessie) system. (64 bit, using g++ 4.9 for compilation)

...

...

After $ cvs -d :pserver:anonymous@fsvm.nmr.mgh.harvard.edu:/usr/fscvsroot login $ cvs -d :pserver:anonymous@fsvm.nmr.mgh.harvard.edu:/usr/fscvsroot checkout -P dev $ cd dev $ cvs update -d # just to make sure $ ./setup_configure

...

...

I get these messages:

...

...

rm -rf autom4te.cache libtoolize --force libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `config'. libtoolize: linking file `config/ltmain.sh' libtoolize: putting macros in `m4'. libtoolize: linking file `m4/libtool.m4' libtoolize: linking file `m4/ltoptions.m4' libtoolize: linking file `m4/ltsugar.m4' libtoolize: linking file `m4/ltversion.m4' libtoolize: linking file `m4/lt~obsolete.m4' libtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.in and libtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree. aclocal aclocal: warning: autoconf input should be named 'configure.ac', not 'configure.in' aclocal: error: configure.in:2045: file 'm4/autotroll.m4' does not exist

...

...

Which looks like a not-(anymore)supported use of autoconf? I found one previous message on the list about autotroll https://mail.nmr.mgh.harvard.edu/pipermail//freesurfer/2012-January/021854.h... that provided another version of configure.in which might solve the issue.

...

...

And it did in the original case but sadly not for me. I get the same error message also with the new configure.in -- there should probably be a configure.ac?

...

...

If anyone knows how to make this work (e.g. by modifying the configure script) that would be fantastic. _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

...

---

Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

...

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

is there a public clone of freesurfer on github?

Sorry, their is no public freesurfer on github. But at the current moment (and forseeable future) the public repo is an exact mirro of the private repo.

--disable-GUI-build -- wouldn't it disable then building all so necessary GUI tools? is there a way to overcome the shortcoming?

I was never able to get the GUI tools to build on Ubuntu. It is something I may revisit in the near future, but as of now, in order to build on an Ubuntu platform you will need to disable GUI building.

We simply dont have the resources to ensure build capability on all linux (and mac) based OSes. For now simply we make the claim that freesurfer will build on centos, and will run on nearly all the major linux distros.

I would be infinitely grateful if freesurfer could build natively on Debian stable! ;) Thanks for trying. In my case I am trying the build under jessie chroot (using schroot helper), so I could quickly setup/enter such environment while maintaining user/directory. I will do the same now for 14.04. If you would like more -- let me know I could give a bit more detailed instructions.

Working on it now. Ill be attempting the build under a vanilla jessie install.

;) it is an output of 'git describe --tags', which bases it on a most recent tag, which was 5622 commits ago and its hexsha f2d0bcd. You could feed this beast to git directly -- it would swallow:

$> git show stable5_branch-5622-gf3d0bcd | head commit f3d0bcd7c4ee8604918725de07c8736191cb1b8f Author: Z K zkaufman@nmr.mgh.harvard.edu Date: Tue Feb 16 17:03:00 2016 -0500

 Changed to reference new GCA. -greve

diff --git a/scripts/recon-all b/scripts/recon-all index 6e4ab62..bf35521 100755 --- a/scripts/recon-all +++ b/scripts/recon-all

Got it! Thanks.

-Zeke

On Sat, 20 Feb 2016, Bruce Fischl wrote:

hmmm, the only thing that worries me about dynamic linking is that it will add variability to the outputs. Zeke has spent endless amounts of time tracking down e.g. mac vs. pc differences in math libs and such. Won't dynamic linking just make that a much more prevalent problem?

nope -- I am not talking about dynamically linking external libraries ATM, but first about the internal ones, where the common code where instead of absorbing the same .a or .o blobs into multiple binaries, there would be a set of internal dynamic libraries which those binaries would be linked against, so there will no copies of binary code among different binaries. I guess https://www.gnu.org/software/automake/manual/html_node/Libtool-Convenience-L... is a quick intro into those.

As for linking against external libraries -- those could be bundled along with rpath pointing to their location or LD_LIBRARY_PATH override assuring they are picked up (instead of possibly available identically named system-wide ones).

So overall it is possible to achieve absent variability while using dynamic linking and allowing for possibility of the flexibility ;)

Cheers

On Sun, 21 Feb 2016, zkaufman@nmr.mgh.harvard.edu wrote:

This topic gets brought up occasionally and their are valid arguments to both sides. One reason we have hesitated to use dynamic libs is the partly due to freesurfers long release cycle (all subjects that are part of a study need to be performed all on the same version). This long release cycle sometimes necessitates fixes to a particular binary in the stream, which users are then free to use. Although this is a less than ideal release strategy, it is the reality of the situation. And if we linked against dynamic libs than any time a binary was updated, ALL those libs, would need to be updated, which in turn would affect all binaries which link against them.

AFAIK it is exactly the other way around ;) Please correct me if I am wrong.

With static inclusion of code, if the fix is in the code which is shared among binaries, you will need to provide new copies for all those effected binaries so they come with new copies of that code. Only if the fix is within code specific to the binary -- only that binary indeed.

In case if that 'fixed' code being a part of an internal dynamic library [*], you would need to provide only a copy of that library, and binaries linked against it can stay original since they just dynamically load that code from the library and do not carry broken code. If it is a fix to the code specific to the binary -- situation is just the same as with static linking.

[*] under assumption that the fix doesn't entail changing API/ABI. In case if those change -- indeed adjustment/rebuild of dependent binaries would be necessary. BUT such situations come much less often than just fixes of the code without changing data structures and function interfaces. And in your case, even if that happens, it is just a matter again of uploading all those affected binaries (as you would do with static linking) + the dynamic library. And again, I think, even if a binary uses some functionality of the library, but not a 'fixed' function, it could as well stay without 'update' while linking to the new dyn library.

That is the primary reason why Linux distributions rely on dynamic libraries and reusing system-wide installed artifacts (e.g. java script "libraries", Python modules, etc) as often as possible -- to fix a vulnerability in the code of a core library/artifact requires just upload of the fixed library/artifact without rebuilding all binaries (or replacing all copies of artifacts) which could have potentially absorbed that code via static "linking". Could you imagine the chaos if libc was statically included in every binary and then security fix was needed to propagate to all 30,000 packages? ;)

I suppose only newly released binaries could be static, but their may be unintended consequences that Im not thinking of at the moment.

Im open to conversing about this, and appreciate any constructive feedback on improving our release model.

Well -- I can't recommend anything new really, and just repeat my whining: modularize just a bit (e.g. separate package for heavy data pieces which rarely change + dynamic libraries, you already have that bundle of externals you could deploy and reuse), and then you can easily release not just "binary patches" but entire bugfix releases of the codebase since they will probably be a fraction of the current size. That is AFAIK how anything is released these days really ;)

NB. somewhat crazier scheme could be -- release binaries via git-annex, with upgrades constituting "git pull; git annex get . " ;) It would become especially efficient in case of dynamic internal libraries, since fetching a bugfix release would be to fetch only affected dynamic libraries and not all affected binaries linking against them.

But I would prefer to take the conversation offline as it gets overly technical rather quick.

ah (I was replying inline without reading all the way till the end, sorry ;)), then this will be the last one then on this mailing list in this series. Could there may be a freesurfer-devel mailing list to discuss such topics? ;)

Hello Yaroslav,

I believe there is already an option that can be set at configure time to build dynamical;ly, though I have not tried it. e.g., maybe it causes the libraries that Freesurfer builds internally to be built as shared instead of static.

However, at least in out nightly builds, the freesurfer binaries link dynamically with the linux system libs. It looks like there are about 130 linux system libraries referenced by all the ELF files in a built freesurfer tree.

By comparison I think there may be no more then 15 static libs built and linked against inside the freesurfer tree. While these could be built shared, I’m not sure there is an expectation that end users should be able to replace/update these libs within a release of Freesiurfer.

- rob

On Mar 29, 2018, at 4:36 PM, Yaroslav Halchenko forum-freesurfer@onerussian.com wrote:

Dear FreeSurfer developers,

I would like to pick up on this elderly theme by wondering:

Would you be philosophically opposed against patches/pull requests which introduce dynamic linking for freesurfer build?

Would anyone from the FreeSurfer team be interested to pursue such an effort?

The idea is to try to reincarnate the effort of pulling functionality/binary-code which is now spread/duplicated across many distributed statically linked binaries, into a set of (internal) libraries. Benefits IMHO are numerous, starting from

• smaller binary distribution

• consistency (replacing one patched binary copy but leaving the

other copies which linked statically unpatched)

• ...

Cheers,

On Sun, 21 Feb 2016, Yaroslav Halchenko wrote:

...

On Sun, 21 Feb 2016, zkaufman@nmr.mgh.harvard.edu wrote:

...

...

This topic gets brought up occasionally and their are valid arguments to both sides. One reason we have hesitated to use dynamic libs is the partly due to freesurfers long release cycle (all subjects that are part of a study need to be performed all on the same version). This long release cycle sometimes necessitates fixes to a particular binary in the stream, which users are then free to use. Although this is a less than ideal release strategy, it is the reality of the situation. And if we linked against dynamic libs than any time a binary was updated, ALL those libs, would need to be updated, which in turn would affect all binaries which link against them.

...

AFAIK it is exactly the other way around ;) Please correct me if I am wrong.

...

With static inclusion of code, if the fix is in the code which is shared among binaries, you will need to provide new copies for all those effected binaries so they come with new copies of that code. Only if the fix is within code specific to the binary -- only that binary indeed.

...

In case if that 'fixed' code being a part of an internal dynamic library [*], you would need to provide only a copy of that library, and binaries linked against it can stay original since they just dynamically load that code from the library and do not carry broken code. If it is a fix to the code specific to the binary -- situation is just the same as with static linking.

...

[*] under assumption that the fix doesn't entail changing API/ABI. In case if those change -- indeed adjustment/rebuild of dependent binaries would be necessary. BUT such situations come much less often than just fixes of the code without changing data structures and function interfaces. And in your case, even if that happens, it is just a matter again of uploading all those affected binaries (as you would do with static linking) + the dynamic library. And again, I think, even if a binary uses some functionality of the library, but not a 'fixed' function, it could as well stay without 'update' while linking to the new dyn library.

...

That is the primary reason why Linux distributions rely on dynamic libraries and reusing system-wide installed artifacts (e.g. java script "libraries", Python modules, etc) as often as possible -- to fix a vulnerability in the code of a core library/artifact requires just upload of the fixed library/artifact without rebuilding all binaries (or replacing all copies of artifacts) which could have potentially absorbed that code via static "linking". Could you imagine the chaos if libc was statically included in every binary and then security fix was needed to propagate to all 30,000 packages? ;)

...

...

I suppose only newly released binaries could be static, but their may be unintended consequences that Im not thinking of at the moment.

...

...

Im open to conversing about this, and appreciate any constructive feedback on improving our release model.

...

Well -- I can't recommend anything new really, and just repeat my whining: modularize just a bit (e.g. separate package for heavy data pieces which rarely change + dynamic libraries, you already have that bundle of externals you could deploy and reuse), and then you can easily release not just "binary patches" but entire bugfix releases of the codebase since they will probably be a fraction of the current size. That is AFAIK how anything is released these days really ;)

...

NB. somewhat crazier scheme could be -- release binaries via git-annex, with upgrades constituting "git pull; git annex get . " ;) It would become especially efficient in case of dynamic internal libraries, since fetching a bugfix release would be to fetch only affected dynamic libraries and not all affected binaries linking against them.

...

...

But I would prefer to take the conversation offline as it gets overly technical rather quick.

...

ah (I was replying inline without reading all the way till the end, sorry ;)), then this will be the last one then on this mailing list in this series. Could there may be a freesurfer-devel mailing list to discuss such topics? ;)

-- Yaroslav O. Halchenko Center for Open Neuroscience http://centerforopenneuroscience.org Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

Intolerable differences have not been found. They are always minor <%5 and their source can come from several places. For example, different compilers (or even different versions of the same compiler) can have different sorting behavior when presented with objects that have equivalent comparators. Others occur with data structures being written/read to disk (Im still trying to fully understand those). As well as a host of other potential differences as you mention below.

However, no cases have been found to fall outside expected and accepted tolerance levels.

-Zeke

On 20 February 2016 at 17:28, Bruce Fischl fischl@nmr.mgh.harvard.edu wrote:

...

hmmm, the only thing that worries me about dynamic linking is that it will add variability to the outputs. Zeke has spent endless amounts of time tracking down e.g. mac vs. pc differences in math libs and such.

What sort of differences were found? Differences in transcendental functions are to be expected (so long as they are small) and lots of other things can cause slightly different results to be generated from floating point calculations (for instance, when summing an array, the result might vary with the number of OpenMP threads used). Were the segmentations produced intolerably different?

Regards,

Richard _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

On Sun, 21 Feb 2016, R Edgar wrote:

On 21 February 2016 at 22:33, zkaufman@nmr.mgh.harvard.edu wrote:

...

Intolerable differences have not been found. They are always minor <%5 and their source can come from several places. For example, different compilers (or even different versions of the same compiler) can have different sorting behavior when presented with objects that have equivalent comparators. Others occur with data structures being written/read to disk (Im still trying to fully understand those). As well as a host of other potential differences as you mention below.

...

However, no cases have been found to fall outside expected and accepted tolerance levels.

That's a relief, but it makes me wonder about another point (and please understand that I'm making the following statement from a position of ignorance as to how Freesurfer is used in practice):

Shouldn't differences like this be embraced? They are giving a measure of the error bars which should be applied to any results.

I'd be interested in hearing people's thoughts about this.

Going back to smaller and concrete things, I'm also curious as to the differences you say are creeping in when IO occurs. Is this happening for formatted or binary I/O?

Before commenting I want to state that this sub-thread is IMHO off-topic, since I don't think it shouldn't (theoretically) have to do anything with dynamic linking of internal code ...

IO issues also Z K mentioned also sounded interesting to my ear ;) smells like alignment issues or uninitialized memory somewhere -- valgrind could be of great help here... eventually, whenever a proper debian package of freesurfer would see a daylight and there would be some unit-tests, building/testing on exotic platforms such as big-endians (powerpc, sparc, s390x) and the ones with stricter alignment requirements (iirc armhf) can help to pin down locations of such bugs since then e.g. 0x1 becomes 0x10000 and thus sticking into your face ;) (we have encounter such ones from time to time with e.g. numpy and scipy)