[Maxima-discuss] lots of test errors with GCL + current Maxima sources

[Robert Dodier]

I've built Maxima from current sources + GCL 2.6.12 (as built from
gcl-2.6.12.tar.gz) and I'm getting hundreds of errors in the test suite
which all seem to be:

Condition in MACSYMA-TOP-LEVEL [or a callee]:
INTERNAL-SIMPLE-TYPE-ERROR: NIL is not of type INTEGER:

Not sure what that's about. Here's an example. I figured out that one
error was in RATTIMES. Loading it means that it will be interpreted and
more debug info is available.

load ("rat3e.lisp");
factor(x^3 - 3*x);
Error:
Fast links are on: do (si::use-fast-links nil) for debugging
Signalled by RATTIMES.
Condition in RATTIMES [or a callee]: INTERNAL-SIMPLE-TYPE-ERROR: NIL is
not of type INTEGER:

At this point I can get a stack trace via :bt. Dunno what to make of it.

I think I was able to build with GCL in the recent past. Has there been
any change that would globally affect GCL?

best

Robert Dodier

[Richard Fateman]

My guess is that GCL is now objecting to a definition like

(defun foo() (let (x) ;;implicitly sets x to nil, type of x is list
(set x 3))

which used to be OK.

A fix would be

(defun foo() (let ((x 0))
(set x 3))

or to make the compiler less obsessive.

That's my guess, anyway.

RJF

Post by Robert Dodier
I've built Maxima from current sources + GCL 2.6.12 (as built from
gcl-2.6.12.tar.gz) and I'm getting hundreds of errors in the test suite
Not sure what that's about. Here's an example. I figured out that one
error was in RATTIMES. Loading it means that it will be interpreted and
more debug info is available.
load ("rat3e.lisp");
factor(x^3 - 3*x);
Fast links are on: do (si::use-fast-links nil) for debugging
Signalled by RATTIMES.
Condition in RATTIMES [or a callee]: INTERNAL-SIMPLE-TYPE-ERROR: NIL is
At this point I can get a stack trace via :bt. Dunno what to make of it.
I think I was able to build with GCL in the recent past. Has there been
any change that would globally affect GCL?
best
Robert Dodier
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Maxima-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/maxima-discuss

[Raymond Toy]

Robert> I've built Maxima from current sources + GCL 2.6.12 (as built from
Robert> gcl-2.6.12.tar.gz) and I'm getting hundreds of errors in the test suite
Robert> which all seem to be:

Same here, with gcl 2.6.12. (Not exactly which 2.6.12, I think I built
this myself some time ago.) Most tests don't seem to run due to an
error break somewhere. I get 1057 tests failed out of 5946 tests. I
know there are like 11000 tests, so more than half aren't run. :-(

I think this really says again that we really do need some kind of
continuous integration that prevents check-ins if the testsuite
doesn't pass. Most people (including me) don't have all the lisps
running locally, so some machine with access to these would be really
halpful.

--
Ray

[Gunter Königsmann]

Post by Raymond Toy
Robert> I've built Maxima from current sources + GCL 2.6.12 (as built from
Robert> gcl-2.6.12.tar.gz) and I'm getting hundreds of errors in the test suite
Same here, with gcl 2.6.12. (Not exactly which 2.6.12, I think I built
this myself some time ago.) Most tests don't seem to run due to an
error break somewhere. I get 1057 tests failed out of 5946 tests. I
know there are like 11000 tests, so more than half aren't run. :-(
I think this really says again that we really do need some kind of
continuous integration that prevents check-ins if the testsuite
doesn't pass. Most people (including me) don't have all the lisps
running locally, so some machine with access to these would be really
halpful.

+1 from me. And since I have managed to break plot and draw completely
for clisp without anybody noting for weeks: An automatic test that looks
if the diagrams gnuplot outputs contain the right kind of graph is next
to impossible. But we could add a few tests to the testbench, perhaps,
that test if a try to produce a 2d or a 3d plot results in a crash or
fails in a similarily horrible way. Also we could perhaps test if all
methods of communicating with gnuplot work: The error I had produced
wasn't reproducible in wxMaxima.
Any objections to that? This step would mean that the testbench will
fail on computers that don't contain gnuplot so adding it might be a
Wrong Decision. And if we want to add it: Are there any ways of
communicating with gnuplot on any platform that are known to fail?

---

I have just seen that https://git.launchpad.net/maxima seems to have
started automatically importing maxima's code again => will try to set
up an automatic build that automatically runs all tests on a daily basis
on launchpad.

Anyone who wants to help with this is welcome in the
https://launchpad.net/~maxima-developers group: I want at least one
second administrator in this group.

---

Another thing I need a hand in: There are reports from 2 users that the
git head of wxMaxima immediately crashes on opening a file or typing any
letter - depending on the computer the program was compiled on.

Unfortunately on my computer it runs fine. If anybody sees this crash:
The output of

gdb wxmaxima
<crash the program>
backtrace

would be really helpful for me.

Thanks a lot,
and
Kind regards,

Gunter.

[Raymond Toy]

Robert> I've built Maxima from current sources + GCL 2.6.12 (as built from
Robert> gcl-2.6.12.tar.gz) and I'm getting hundreds of errors in the test suite

Post by Raymond Toy
Same here, with gcl 2.6.12. (Not exactly which 2.6.12, I think I built
this myself some time ago.) Most tests don't seem to run due to an
error break somewhere. I get 1057 tests failed out of 5946 tests. I
know there are like 11000 tests, so more than half aren't run. :-(
I think this really says again that we really do need some kind of
continuous integration that prevents check-ins if the testsuite
doesn't pass. Most people (including me) don't have all the lisps
running locally, so some machine with access to these would be really
halpful.

Gunter> +1 from me. And since I have managed to break plot and draw completely
Gunter> for clisp without anybody noting for weeks: An automatic test that looks
Gunter> if the diagrams gnuplot outputs contain the right kind of graph is next
Gunter> to impossible. But we could add a few tests to the testbench, perhaps,

Well, we could have the test save the plots in png (or other format),
and compare the new output with some previously saved output and note
the difference. If they're different, the submitter will need to look
at the difference and decide if it's ok and as part of the change,
update new expected results.

This is a giant pain.

Gunter> that test if a try to produce a 2d or a 3d plot results in a crash or
Gunter> fails in a similarily horrible way. Also we could perhaps test if all
Gunter> methods of communicating with gnuplot work: The error I had produced
Gunter> wasn't reproducible in wxMaxima.
Gunter> Any objections to that? This step would mean that the testbench will
Gunter> fail on computers that don't contain gnuplot so adding it might be a
Gunter> Wrong Decision. And if we want to add it: Are there any ways of
Gunter> communicating with gnuplot on any platform that are known to fail?

While gnuplot is actually pretty much separate from maxima itself, I,
for one, need gnuplot to work when I'm using maxima.

Gunter> I have just seen that https://git.launchpad.net/maxima seems to have
Gunter> started automatically importing maxima's code again => will try to set
Gunter> up an automatic build that automatically runs all tests on a daily basis
Gunter> on launchpad.

This is cool, but unless the integration is tied to Sourceforge so
that test failures prevent checkins, this doesn't really help. It has
to be automatic and enforced. People who don't check today (including
me!) likely won't check other sites to see if the commit was ok.

--
Ray

[Gunter Königsmann]

Post by Raymond Toy
Robert> I've built Maxima from current sources + GCL 2.6.12 (as built from
Robert> gcl-2.6.12.tar.gz) and I'm getting hundreds of errors in the test suite

Post by Raymond Toy
Same here, with gcl 2.6.12. (Not exactly which 2.6.12, I think I built
this myself some time ago.) Most tests don't seem to run due to an
error break somewhere. I get 1057 tests failed out of 5946 tests. I
know there are like 11000 tests, so more than half aren't run. :-(
I think this really says again that we really do need some kind of
continuous integration that prevents check-ins if the testsuite
doesn't pass. Most people (including me) don't have all the lisps
running locally, so some machine with access to these would be really
halpful.

Gunter> +1 from me. And since I have managed to break plot and draw completely
Gunter> for clisp without anybody noting for weeks: An automatic test that looks
Gunter> if the diagrams gnuplot outputs contain the right kind of graph is next
Gunter> to impossible. But we could add a few tests to the testbench, perhaps,
Well, we could have the test save the plots in png (or other format),
and compare the new output with some previously saved output and note
the difference. If they're different, the submitter will need to look
at the difference and decide if it's ok and as part of the change,
update new expected results.
This is a giant pain.

This would add the whole chain of gnuplot, libcairo etc., to the test,
too => we would need to handle the case that every user gets a different
result for the same plot.

But perhaps besides testing if trying to plot crashes we could test if
the input files we produce for gnuplot are the same as normally.

Post by Raymond Toy
Gunter> that test if a try to produce a 2d or a 3d plot results in a crash or
Gunter> fails in a similarily horrible way. Also we could perhaps test if all
Gunter> methods of communicating with gnuplot work: The error I had produced
Gunter> wasn't reproducible in wxMaxima.
Gunter> Any objections to that? This step would mean that the testbench will
Gunter> fail on computers that don't contain gnuplot so adding it might be a
Gunter> Wrong Decision. And if we want to add it: Are there any ways of
Gunter> communicating with gnuplot on any platform that are known to fail?
While gnuplot is actually pretty much separate from maxima itself, I,
for one, need gnuplot to work when I'm using maxima.

This is what I hoped for.

Post by Raymond Toy
Gunter> I have just seen that https://git.launchpad.net/maxima seems to have
Gunter> started automatically importing maxima's code again => will try to set
Gunter> up an automatic build that automatically runs all tests on a daily basis
Gunter> on launchpad.
This is cool, but unless the integration is tied to Sourceforge so
that test failures prevent checkins, this doesn't really help. It has
to be automatic and enforced. People who don't check today (including
me!) likely won't check other sites to see if the commit was ok.

launchpad can be instructed to send Mails if the build or the test
fails. We could subscribe launchpad to a new mailing list on sourceforge
so the members of the sourceforge project can subscribe to these mails
if they want to.

Preventing checkins would be nice. But I currently don't know of a
solution-as-a-service for that that we can use for free.
When Andrej has set up a travis build for wxMaxima it has improved
things a lot, though, by telling that something has failed a few minutes
after it was committed, though.

Kind regards,

Gunter.

[Gunter Königsmann]

Post by Gunter Königsmann
launchpad can be instructed to send Mails if the build or the test
fails. We could subscribe launchpad to a new mailing list on sourceforge
so the members of the sourceforge project can subscribe to these mails
if they want to.

Ok... ...the launchpad build works on one Ubuntu release
(https://launchpadlibrarian.net/328282532/buildlog.txt.gz), but not for
another (https://launchpadlibrarian.net/328282469/buildlog.txt.gz) which
isn't our fault at all

=> deactivating the build for Ubuntu Artful.

On the plus side we now have a nightly build for 32 and 64 Bit Intel
architectures and for the next builds I'll try to enable the ARM Build, too.

If any admin is willing to create a maxima-ci mailing list on
sourceforge.net we can subscribe this list for mails from the launchpad
build demon and any of the subscribers of this list will get mails if
any of the platforms fails (unfortunately we will get one mail per
platform that fails for every day that sees commits that don't fix this
problem ;-( )

If anyone knows how to create a build with more lisps on Ubuntu the
launchpad build demon follows the instructions from the branch
DebianPackaging in our main maxima repo. So for changing these things no
launchpad account is needed at all.

---

Launchpad tells me that there are maxima versions available with version
numbers > 5.40.0-<the current date>:
https://code.launchpad.net/~maxima-developers/+archive/ubuntu/maxima-nightly
Is this logical at all and do we want to increase our version number
somewhere?

Currently I am on mobile data only so I cannot do much testing. Also I
have no idea if the nightly build actually installs, works and if it
breaks the whole debian infrastructure as it did the last time. But at
least we now have continuous integration.

Kind regards,

Gunter.

[Gunter Königsmann]

Seems like the build works but the packaging still not.
Will investigate in the next few days.

Kind regards,

Gunter.

[Robert Dodier]

Post by Gunter Königsmann
This would add the whole chain of gnuplot, libcairo etc., to the test,
too => we would need to handle the case that every user gets a different
result for the same plot.
But perhaps besides testing if trying to plot crashes we could test if
the input files we produce for gnuplot are the same as normally.

Maybe it's enough to just arrange a collection of plotting examples
and their expected output, in some way that makes it easy to generate
the outputs from a new version of Maxima and compare them (for some
value of "compare") with the expected. Then anybody who has a different
combination of Gnuplot, libraries, etc can try it.

About comparing outputs, I have a mental image of being able to see the
succession of outputs for a given example all together. E.g. something
like an html page <img src="example1_2017_07_11.png"/> <img
src=example1_2017_07_15.png"/> ... (or naming by Git commit or
whatever). It seems more relevant to group by example than by date
or version.

Also about comparing, maybe generate SVG outputs -- these are plain text
(XML) files. Or maybe just compare the Gnuplot outputs. Testing these
files can/should be automated.

best

Robert Dodier

[Jaime Villate]

Post by Robert Dodier
Maybe it's enough to just arrange a collection of plotting examples
and their expected output, in some way that makes it easy to generate
the outputs from a new version of Maxima and compare them (for some
value of "compare") with the expected. Then anybody who has a different
combination of Gnuplot, libraries, etc can try it.

I fully agree. In fact, I think this discussion has come up in the past
and I explained that every time I made any changes to plot* I run all
the examples in the manual, with various different Lisps, and "look" at
the results to make sure they give the expected plots, before committing
changes.

Regards,

Jaime

[Gunter Königsmann]

Post by Jaime Villate
and I explained that every time I made any changes to plot* I run all
the examples in the manual, with various different Lisps, and "look" at
the results to make sure they give the expected plots, before
committing
changes.

Last time the error could only be reproduced when wxMaxima wasn't used. So this is a good start but no test is perfect.
Unfortunately several of the plot examples (I unfortunately don't remember if only ones involving descriptive) currently don't work.
Kind regards,

Gunter.

[Jaime Villate]

Post by Gunter Königsmann
Last time the error could only be reproduced when wxMaxima wasn't used. So this is a good start but no test is perfect.
Unfortunately several of the plot examples (I unfortunately don't remember if only ones involving descriptive) currently don't work.

As I said, last time I committed any changes to plot(2d,3d, df) all of
the examples worked :)

Jaime

[Robert Dodier]

Post by Jaime Villate
I fully agree. In fact, I think this discussion has come up in the past
and I explained that every time I made any changes to plot* I run all
the examples in the manual, with various different Lisps, and "look" at
the results to make sure they give the expected plots, before committing
changes.

I think something that could help is some kind of side by side
comparison of the actual output with the expected. I don't really know
what a lot of the plotting examples are supposed to do. Also, there are
a lot of examples in tests/rtest_plot.mac which are not in the
manual -- it would be great to verify those too.

best

Robert Dodier

[Raymond Toy]

Post by Robert Dodier
Maybe it's enough to just arrange a collection of plotting examples
and their expected output, in some way that makes it easy to generate
the outputs from a new version of Maxima and compare them (for some
value of "compare") with the expected. Then anybody who has a different
combination of Gnuplot, libraries, etc can try it.

Jaime> I fully agree. In fact, I think this discussion has come up in the
Jaime> past and I explained that every time I made any changes to plot* I run
Jaime> all the examples in the manual, with various different Lisps, and
Jaime> "look" at the results to make sure they give the expected plots,
Jaime> before committing changes.

That is totally awesome that you do that!

Many thanks for doing that hard work!

--
Ray

[Raymond Toy]

Robert> I've built Maxima from current sources + GCL 2.6.12 (as built from
Robert> gcl-2.6.12.tar.gz) and I'm getting hundreds of errors in the test suite

Post by Raymond Toy

Post by Raymond Toy
Same here, with gcl 2.6.12. (Not exactly which 2.6.12, I think I built
this myself some time ago.) Most tests don't seem to run due to an
error break somewhere. I get 1057 tests failed out of 5946 tests. I
know there are like 11000 tests, so more than half aren't run. :-(
I think this really says again that we really do need some kind of
continuous integration that prevents check-ins if the testsuite
doesn't pass. Most people (including me) don't have all the lisps
running locally, so some machine with access to these would be really
halpful.

Gunter> +1 from me. And since I have managed to break plot and draw completely
Gunter> for clisp without anybody noting for weeks: An automatic test that looks
Gunter> if the diagrams gnuplot outputs contain the right kind of graph is next
Gunter> to impossible. But we could add a few tests to the testbench, perhaps,

Post by Raymond Toy
Well, we could have the test save the plots in png (or other format),
and compare the new output with some previously saved output and note
the difference. If they're different, the submitter will need to look
at the difference and decide if it's ok and as part of the change,
update new expected results.
This is a giant pain.

Gunter> This would add the whole chain of gnuplot, libcairo etc., to the test,
Gunter> too => we would need to handle the case that every user gets a different
Gunter> result for the same plot.

Yes and no. The part that really matters is if the machine(s) doing
the continuous integration produces consistent results. It's annoying
if the local results don't match. Not sure what to do about that case.

Gunter> that test if a try to produce a 2d or a 3d plot results in a crash or
Gunter> fails in a similarily horrible way. Also we could perhaps test if all
Gunter> methods of communicating with gnuplot work: The error I had produced
Gunter> wasn't reproducible in wxMaxima.
Gunter> Any objections to that? This step would mean that the testbench will
Gunter> fail on computers that don't contain gnuplot so adding it might be a
Gunter> Wrong Decision. And if we want to add it: Are there any ways of
Gunter> communicating with gnuplot on any platform that are known to fail?

Since gnuplot isn't strictly required for maxima, I would not like
this to be a hard requirement for the tests. So maybe not part of
run_testsuite(), but ok for share tests (or something else).

--
Ray

[Gunter Königsmann]

I have added a wxMaxima worksheet named tests/draw.wxm that is a
beginning for a potential test bench for draw.
I haven't finished the text waiting for comments for the list. But it
seems that many of the tests currently fail. Haven't looked at the
details, though.

Now I have added comments to all the plots in tests/draw.wxm that
actually worked. Seems like most of the geomap examples are broken,
currently. But as others are broken, too, they all might be a symptom on
a single change at a completely different place.

Kind regards,

Gunter.

[Mario Rodriguez]

Post by Gunter Königsmann

I have added a wxMaxima worksheet named tests/draw.wxm that is a
beginning for a potential test bench for draw.
I haven't finished the text waiting for comments for the list. But it
seems that many of the tests currently fail. Haven't looked at the
details, though.

Now I have added comments to all the plots in tests/draw.wxm that
actually worked. Seems like most of the geomap examples are broken,
currently. But as others are broken, too, they all might be a symptom on
a single change at a completely different place.
Kind regards,
Gunter.

Thanks for your efforts, Gunter.

I have taken a look at the errors. Here are some suggestions:

* Write kill(all) just before draw_proportional_axis. Variable xy was
bounded to a variable in a previous test and Maxima doesn't understand
proportional_axis=xy. You can also write proportional_axis='xy to
prevent this error. Similar problems with variables x, y, a, and z.

* Remove example world_map_geomap. It breakes everything after it. This
example modifies boundaries_array and new loads of worldmap don't
restore the boundaries. In fact, this example was removed from my web
site some time ago.

* Finally, rewrite example worldmap_region_boundaries_plus and don't use
%, I mean something like this:
rbp: region_boundaries_plus(10.4,41.5,20.7,35.4);
docdraw2d("figures/worldmap_region_boundaries_plus",geomap(rbp))$

Hope this all helps.

[Gunter Königsmann]

Dear Mario,

Now the manual draw testbench nearly runs. But it still runs into a few
errors.

Kind regards,

Gunter.

[Mario Rodriguez]

Post by Gunter Königsmann
Dear Mario,
Now the manual draw testbench nearly runs. But it still runs into a few
errors.
Kind regards,
Gunter.

Write kill(a) before test draw_spherical or single quote a in
draw_spherical.mac. The other errors are of similar nature.

[Robert Dodier]

Post by Raymond Toy
This is cool, but unless the integration is tied to Sourceforge so
that test failures prevent checkins, this doesn't really help. It has
to be automatic and enforced. People who don't check today (including
me!) likely won't check other sites to see if the commit was ok.

I'm not sure if we should want to prevent a commit when a build fails.
It might be preferable to let the offending commit stand -- e.g. when
someone other than the commit author can fix it.

For my part, I'd be satisfied if one could check the list of builds from
time to time -- e.g. before making a release. Just my $0.02.

best

Robert Dodier

[Viktor T. Toth]

For what it's worth, when I am making a release, I first build using six different LISPs (SBCL/GCL/CMUCL/CCL/ECL/CLISP). If there are unexpected errors, I initiate a discussion and delay the release if necessary.

This does not guarantee of course that there won't be errors on other configurations, but being able to build on a mainstream platform (CentOS) with many different LISPs offers a level of assurance.


Viktor

-----Original Message-----
Sent: Tuesday, July 11, 2017 11:39 PM
Subject: [Maxima-discuss] automated build testing, was: lots of test errors with GCL + current Maxima sources

Post by Raymond Toy
This is cool, but unless the integration is tied to Sourceforge so
that test failures prevent checkins, this doesn't really help. It has
to be automatic and enforced. People who don't check today (including
me!) likely won't check other sites to see if the commit was ok.

I'm not sure if we should want to prevent a commit when a build fails.
It might be preferable to let the offending commit stand -- e.g. when
someone other than the commit author can fix it.
For my part, I'd be satisfied if one could check the list of builds from
time to time -- e.g. before making a release. Just my $0.02.
best
Robert Dodier
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Maxima-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/maxima-discuss

[Gunter Königsmann]

@Jaime: I didn't doubt that it is possible to test all the draw
examples. But I doubt that everybody searches for them in the manual.
Also I doubt that in this process the descriptive() ones will frequently
be found as they don't contain a literal plot() command...
...and (as plot isn't changed frequently and not always by a person
owning the newest gnuplot) the errors might be due to accumulated bitrot
and not due to a change in plot, anyway.

In the meantime I have made tests/draw.wxm which contains all the draw
examples from the documentation work.

The next question would be: Should I change this file in a way that it
is usable in plain maxima without wxMaxima?
The next step might be adding all the plot commands from the manual:
If tests aren't too hard to run this increases the probability that they
will.

Kind regards,

Gunter.

[Robert Dodier]

Post by Gunter Königsmann
In the meantime I have made tests/draw.wxm which contains all the draw
examples from the documentation work.
The next question would be: Should I change this file in a way that it
is usable in plain maxima without wxMaxima?
If tests aren't too hard to run this increases the probability that they
will.

Well, there is already tests/rtest_plot.mac which contains examples from
the manual as well as other examples not in the manual. So if you have
some new examples maybe you can add them to rtest_plot.mac.

What we're lacking is a way to compare the actual and expected output
visually. I see that one can say

gnuplot> plot 'path/to/my.png' binary filetype=png with rgbimage

to display an image file in Gnuplot. Maybe somehow display the plot and
the image at the same time? No doubt we can think of several ways to
accomplish that.

best,

Robert Dodier

[Kris Katterjohn]

Post by Gunter Königsmann
The next question would be: Should I change this file in a way that it
is usable in plain maxima without wxMaxima?

I've been busy and haven't been following this discussion closely (so
perhaps I've misunderstood something), but I don't think that tests
should depend on a particular front-end (wxmaxima, xmaxima, etc.) unless
there are really good reasons to do so.

Not everyone uses wxmaxima and it resides in a completely separate repo.
Xmaxima is included in our repo, but not everyone uses it so I don't
think tests should depend on it either (again, unless there are good
reasons to do so).

I admit that I haven't thought much about a solution, but I do think it
would be best to try to avoid depending on a particular front-end for
tests. Let me know if I've misunderstood something.

Cheers,
Kris Katterjohn

[Gunter Königsmann]

Post by Kris Katterjohn
I admit that I haven't thought much about a solution, but I do think it
would be best to try to avoid depending on a particular front-end for
tests. Let me know if I've misunderstood something.

You didn't misunderstand anything.

Since your argument that the
testbench should be independent of the front-end being in use is valid I
have now changed test/draw.wxm in a way that if it is being executed
from wxMaxima it will form a worksheet with embedded plots, while if it
is loaded from maxima it will produce standalone plots instead and wait
for the user to press <enter> between plots.

I bet things it can be improved in various ways (for example with
Robert's idea of showing "good" pictures besides the one we produce)
but... ...that is how far I got.

Kind regards,

Gunter.

[Wolfgang Dautermann]

Post by Viktor T. Toth
For what it's worth, when I am making a release, I first build using
six different LISPs (SBCL/GCL/CMUCL/CCL/ECL/CLISP).

And additionally there are the automated nightly tests using 8 different
LISPs at: http://wolfgang.dautermann.at/maxima/nightlybuild/

(Remark: They will probably not build between the 28th of July and the
10th of August).

Post by Viktor T. Toth
If there are unexpected errors, I initiate a discussion and delay the
release if necessary.

Just curious: I get an unexpected error with ECL since some time:
http://wolfgang.dautermann.at/maxima/nightlybuild/logfile-testsuite-ecl.txt

This is one of the LISPs you test too. Does it work for you without
errors? (I use Ubuntu 16.04 with an up-to-date ECL installed in
/opt/ecl-16.1.3)

Best regards, Wolfgang

[Robert Dodier]

Post by Wolfgang Dautermann
http://wolfgang.dautermann.at/maxima/nightlybuild/logfile-testsuite-ecl.txt

Some experimentation shows that li[2](1.0*%i) => division by zero is
caused by (log #C(-0.0 -1.0)) => division by zero in Lisp. I see that's
been reported to the ECL project and fixed (post-16.3.1):
https://gitlab.com/embeddable-common-lisp/ecl/issues/329

I don't know if it was discussed on the Maxima mailing list -- I don't
remember, at any rate.

best

Robert Dodier

[Kris Katterjohn]

Post by Robert Dodier

Post by Wolfgang Dautermann
http://wolfgang.dautermann.at/maxima/nightlybuild/logfile-testsuite-ecl.txt

Some experimentation shows that li[2](1.0*%i) => division by zero is
caused by (log #C(-0.0 -1.0)) => division by zero in Lisp. I see that's
https://gitlab.com/embeddable-common-lisp/ecl/issues/329
I don't know if it was discussed on the Maxima mailing list -- I don't
remember, at any rate.

For completeness, it was discussed here:

https://sourceforge.net/p/maxima/mailman/message/35559566/

Cheers,
Kris Katterjohn

[Raymond Toy]

Post by Raymond Toy
This is cool, but unless the integration is tied to Sourceforge so
that test failures prevent checkins, this doesn't really help. It has
to be automatic and enforced. People who don't check today (including
me!) likely won't check other sites to see if the commit was ok.

Robert> I'm not sure if we should want to prevent a commit when a build fails.
Robert> It might be preferable to let the offending commit stand -- e.g. when
Robert> someone other than the commit author can fix it.

Of course I'd want a way to allow that. For the few systems I've
used, the changes are on a branch, and the build results are done
there. I don't think anything actually prevents people from merging
that branch to master.

Robert> For my part, I'd be satisfied if one could check the list of builds from
Robert> time to time -- e.g. before making a release. Just my $0.02.

Given that it's 4 months between releases, there could be tons of
changes that nobody will notice for 4 months. And everyone is too
lazy (including me!), for the most part, to do the bisect and hard
work to figure out what broke. With limited manpower, CI is the best
way to go, if we can.

I think sourceforge has the necessary web hooks to support continuous
integration, but I haven't seen any instructions on how to do that.
Which is really sad since github and gitlab and bitbucket all support
continuous integration integrated with revision control.

If I could find some reasonable instructions I would definitely try to
hook up CI for us.

--
Ray

[Andreas Eder]

Post by Raymond Toy
Robert> I've built Maxima from current sources + GCL 2.6.12 (as built from
Robert> gcl-2.6.12.tar.gz) and I'm getting hundreds of
errors in the test suite
Same here, with gcl 2.6.12. (Not exactly which 2.6.12, I think I built
this myself some time ago.) Most tests don't seem to run due to an
error break somewhere. I get 1057 tests failed out of 5946
tests. I
know there are like 11000 tests, so more than half aren't run.
:-(

I don't have gcl-2.6.12.
I'm running GCL (GNU Common Lisp) 2.6.10 ANSI Apr 2 2014
14:25:44
and the testsuite runs without any errors at all.
That is just another data point I know.
I don't know what makes gcl-2.6.12 generate so many errors.

Andreas

[Andreas Eder]

Meanwhile I managed to install gcl 2.6.12;
GCL (GNU Common Lisp) 2.6.12 ANSI Jul 10 2017 22:51:31
Source License: LGPL(gcl,gmp), GPL(unexec,bfd,xgcl)

And the testsuite still runs without any errors.

What are you doing? What system?

I am running Linux iona 3.13.0-124-lowlatency #173-Ubuntu SMP
PREEMPT
Thu Jun 29 17:58:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux.

Seems to be a mysterious problem.

Andreas

[Raymond Toy]

Andreas> Meanwhile I managed to install gcl 2.6.12;
Andreas> GCL (GNU Common Lisp) 2.6.12 ANSI Jul 10 2017 22:51:31
Andreas> Source License: LGPL(gcl,gmp), GPL(unexec,bfd,xgcl)

Andreas> And the testsuite still runs without any errors.

Weird. I'm pretty sure it used to pass, but I don't regularly run
against gcl so I can't say for sure anymore.

Andreas> What are you doing? What system?

Nothing special. Just do run_testsuite() with gcl. Running on an
ubuntu 14.04 system.


Andreas> I am running Linux iona 3.13.0-124-lowlatency #173-Ubuntu SMP PREEMPT
Andreas> Thu Jun 29 17:58:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux.

Andreas> Seems to be a mysterious problem.

Indeed.

Could be that I built gcl myself, but, as I said, I'm pretty sure the
testsuite used to pass with this build. But I also changed my machine
to a different one recently, and just copied over my existing build of
gcl. Maybe that broke something even though the OS is supposed to be
exactly the same. I'm finding other little things on this new mahcine
that is different from before. Yuck.

--
Ray

[Thomas D. Dean]

I am missing something here. What?

Ubuntu 16.04.

I use gcl 12.6.12 built on this machine from the tarball.
I build maxima from git head... Maybe a couple weeks ago.

Post by Robert Dodier
maxima

Maxima branch_5_40_base_80_ga55f0b7 http://maxima.sourceforge.net
using Lisp GNU Common Lisp (GCL) GCL 2.6.12
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1) run_testsuite();
Running tests in rtest_rules: 103/103 tests passed
Running tests in rtestnset: 601/601 tests passed
Running tests in rtest1: 184/184 tests passed (not counting 3 expected
errors)
Running tests in rtest1a: 34/34 tests passed (not counting 1 expected
errors)
Running tests in rtest2: 261/261 tests passed (not counting 2 expected
errors)
...
Running tests in rtest_plotoptions: 5/5 tests passed
Running tests in rtest_algsys: 55/55 tests passed


No unexpected errors found out of 11,227 tests.
real time : 100.139 secs
run-gbc time : 75.500 secs
child run time : 9.770 secs
gbc time : 12.710 secs

Tom Dean

[Thomas D. Dean]

Post by Thomas D. Dean
I am missing something here. What?
Ubuntu 16.04.
I use gcl 12.6.12 built on this machine from the tarball.
I build maxima from git head... Maybe a couple weeks ago.

Post by Robert Dodier
maxima

Maxima branch_5_40_base_80_ga55f0b7 http://maxima.sourceforge.net
using Lisp GNU Common Lisp (GCL) GCL 2.6.12
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1) run_testsuite();
Running tests in rtest_rules: 103/103 tests passed
Running tests in rtestnset: 601/601 tests passed
Running tests in rtest1: 184/184 tests passed (not counting 3 expected
errors)
Running tests in rtest1a: 34/34 tests passed (not counting 1 expected
errors)
Running tests in rtest2: 261/261 tests passed (not counting 2 expected
errors)
...
Running tests in rtest_plotoptions: 5/5 tests passed
Running tests in rtest_algsys: 55/55 tests passed
No unexpected errors found out of 11,227 tests.
real time : 100.139 secs
run-gbc time : 75.500 secs
child run time : 9.770 secs
gbc time : 12.710 secs
Tom Dean

I just updated to git head and re-built. Now, I get errors. The
problem came up between

Maxima branch_5_40_base_80_ga55f0b7 http://maxima.sourceforge.net
and
Maxima branch_5_40_base_121_g335843b http://maxima.sourceforge.net

Post by Thomas D. Dean
maxima

Maxima branch_5_40_base_121_g335843b http://maxima.sourceforge.net
using Lisp GNU Common Lisp (GCL) GCL 2.6.12
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1) run_testsuite();
Running tests in rtest_rules: 103/103 tests passed
Running tests in rtestnset: 601/601 tests passed
Running tests in rtest1: 184/184 tests passed (not counting 3 expected
errors)
Running tests in rtest1a: 34/34 tests passed (not counting 1 expected
errors)
Running tests in rtest2: 261/261 tests passed (not counting 2 expected
errors)
Running tests in rtest4: 93/93 tests passed
Running tests in rtest5:
********************** Problem 15 (line 44) ***************
Input:
1/2 1/4
expand(rootscontract(x y ) - sqrt(x sqrt(y)), 0, 0)


Result:
error-catch

This differed from the expected result:
0

********************** Problem 16 (line 47) ***************
Input:
1/2 1/3 1/3
expand(rootscontract(x y ) - sqrt(x) y , 0, 0)


Result:
error-catch

This differed from the expected result:
0

...

********************** Problem 192 (line 893) ***************
Input:
3/2
(sqrt(3) %i + 1) - 4 %i
rootscontract(--------------------------)
sqrt(sqrt(3) %i + 1)


Result:
error-catch

This differed from the expected result:
3/2
(sqrt(- 3) + 1) - 4 %i
-------------------------
sqrt(sqrt(- 3) + 1)

Things seem to hang here. If I enter ';' I get back to the prompt.

I see lots of difference in rootscontract.

Tom Dean

[Andreas Eder]

This is weird!

I'm runnning GCL 2.6.12 (built from the tarball) and Maxima
branch_5_40_base_121_g335843b (git head) and I don't get any
problems.
The testsuite runs fine.

It does so with gcl, sbcl clisp and ccl64.
Only ecl gives me a failure.
But that has to to with floating point accuracy. (Problem 50, line
268
in rtest16.mac).

Andreas

[Volker van Nek]

I compiled the sources from maxima-5.40.0.tar.gz and current git.
In both cases I got one error in rtest16.
I used

GCL (GNU Common Lisp) 2.6.12 ANSI Oct 29 2015 23:22:33

I guess there exist different versions of GCL 2.6.12.

Volker van Nek

[Gunter Königsmann]

Post by Volker van Nek
I compiled the sources from maxima-5.40.0.tar.gz and current git.
In both cases I got one error in rtest16.
I used
GCL (GNU Common Lisp) 2.6.12 ANSI Oct 29 2015 23:22:33
I guess there exist different versions of GCL 2.6.12.

On my system the 53th 2.6.12 release for Ubuntu is installed. I guess at
least half of these 2.6.12 releases change the code and not only the
packaging. One that has declared many methods private has broken
wxMaxima. For debian I have still to find out how to upload the fix.
Other gcl fixes have repeatedly broken draw.

Kind regards,

Gunter.

[Thomas D. Dean]

Post by Gunter Königsmann

Post by Volker van Nek
I compiled the sources from maxima-5.40.0.tar.gz and current git.
In both cases I got one error in rtest16.
I used
GCL (GNU Common Lisp) 2.6.12 ANSI Oct 29 2015 23:22:33
I guess there exist different versions of GCL 2.6.12.

On my system the 53th 2.6.12 release for Ubuntu is installed. I guess at
least half of these 2.6.12 releases change the code and not only the
packaging. One that has declared many methods private has broken
wxMaxima. For debian I have still to find out how to upload the fix.
Other gcl fixes have repeatedly broken draw.

I have problems with several of the Ubuntu GCL builds. I build from the
GNU 2.6.12 tarball. That always seems to work.

Tom Dean

[Robert Dodier]

Post by Robert Dodier
I've built Maxima from current sources + GCL 2.6.12 (as built from
gcl-2.6.12.tar.gz) and I'm getting hundreds of errors in the test suite

It appears that the problem stems from incorrect code generated for
src/lesfac.lisp -- after load("lesfac.lisp"); (so the code is
interpreted) the test suite runs without all those errors.

I suspect commit a55f0b7 (declare ignore some variables in FACRTIMES)
tickles the bug.

I'm still OK with non-functional changes to the code, but I would like
to take this opportunity to emphasize the responsibility of the authors
of such changes to test such changes thoroughly, including with multiple
Lisp implementations.

best

Robert Dodier

[Andreas Eder]

On Di 11 Jul 2017 at 08:03, Robert Dodier

Post by Robert Dodier
It appears that the problem stems from incorrect code generated for
src/lesfac.lisp -- after load("lesfac.lisp"); (so the code is
interpreted) the test suite runs without all those errors.
I suspect commit a55f0b7 (declare ignore some variables in
FACRTIMES)
tickles the bug.
I'm still OK with non-functional changes to the code, but I
would like
to take this opportunity to emphasize the responsibility of the authors
of such changes to test such changes thoroughly, including with multiple
Lisp implementations.

If it is indeed that change, I did that and I tested it with gcl,
ecl,
sbcl .ccl and clisp. And it worked in all cases.
I did the test both with gcl version 2.6.10 and 2.6.12 without any
problems.
So it seems that you have a very special 2.6.12 version of gcl.

best

Andreas

[Gunter Königsmann]

If it is indeed that change, I did that and I tested it with gcl, ecl,
sbcl .ccl and clisp. And it worked in all cases.
I did the test both with gcl version 2.6.10 and 2.6.12 without any
problems.
So it seems that you have a very special 2.6.12 version of gcl.

gcl didn't increase the version number in the last few years. But every
Ubuntu version (and I guess every debian version) since the release of
2.6.12 incorporates several bug fixes. Which is a feature that in case
that a package's official release has a horrible bug allows debian to
provide a 2.6.12-1 or a 2.6.12-5 package that incorporates bugfixes.
Unfortunately the bugfixes won't change the version number the affected
program thinks it has.

Kind regards,

Gunter.

[Thomas D. Dean]

Post by Gunter Königsmann

If it is indeed that change, I did that and I tested it with gcl, ecl,
sbcl .ccl and clisp. And it worked in all cases.
I did the test both with gcl version 2.6.10 and 2.6.12 without any
problems.
So it seems that you have a very special 2.6.12 version of gcl.

gcl didn't increase the version number in the last few years. But every
Ubuntu version (and I guess every debian version) since the release of
2.6.12 incorporates several bug fixes. Which is a feature that in case
that a package's official release has a horrible bug allows debian to
provide a 2.6.12-1 or a 2.6.12-5 package that incorporates bugfixes.
Unfortunately the bugfixes won't change the version number the affected
program thinks it has.

Looking at gcl for several Ubuntu releases, copied from debian:

14.04 is using 2.6.10, very old
16.04 is using 2.6.12-29, somewhere out in the development chain
16.10 is using 2.6.12-33, somewhere out in the development chain
17.04 is using 2.6.12-47, somewhere out in the development chain
17.10 is using 2.6.12-53, somewhere out in the development chain

The development chain seems to mostly be developed by Camm Maguire,
***@debian.org, a gcl developer. Maybe the only one???

The last release of GNU GCL is gcl-2.6.12.tar.gz dated 28-Oct-2014.

There is a 9 month old Version_2_6_13pre in git. Maybe a release is
pending in a year, or so?

Tom Dean

[Robert Dodier]

If it is indeed that change, I did that and I tested it with gcl, ecl,
sbcl .ccl and clisp. And it worked in all cases.
I did the test both with gcl version 2.6.10 and 2.6.12 without any
problems.

OK, sounds good. That's as much as we can hope for.

So it seems that you have a very special 2.6.12 version of gcl.

Well, I built it from the gcl-2.6.12.tar.gz which is the most recent
tarball at the GCL project page (savannah.gnu.org or whatever it is).
But that tar file is a few years old already, and I guess there have
been changes since then. It is unfortunate that different versions
of GCL advertise the same version number. It seems desirable
to label a new version of GCL; I wonder if anyone can help to make
it happen.

best,
Robert Dodier

[Andreas Eder]

On Di 11 Jul 2017 at 18:48, Robert Dodier

On Tue, Jul 11, 2017 at 8:47 AM, Andreas Eder

Post by Andreas Eder
So it seems that you have a very special 2.6.12 version of gcl.

Well, I built it from the gcl-2.6.12.tar.gz which is the most
recent
tarball at the GCL project page (savannah.gnu.org or whatever it is).
But that tar file is a few years old already, and I guess there have
been changes since then. It is unfortunate that different
versions
of GCL advertise the same version number. It seems desirable
to label a new version of GCL; I wonder if anyone can help to
make
it happen.

But that is even more strange, because that is presumably the same
tarball that I built gcl from. So it should be the same gcl ?!

best

Andreas

[Robert Dodier]

Post by Andreas Eder
But that is even more strange, because that is presumably the same
tarball that I built gcl from. So it should be the same gcl ?!

Yeah, it should be the same. I don't know what's going on.

best,

Robert Dodier