Discussion:
Side effects of translating macro definitions
(too old to reply)
Kris Katterjohn
2017-06-13 23:26:56 UTC
Permalink
Raw Message
Hey everyone,

I've noticed that translating a function that contains a macro
definition causes the macro to actually be defined. This seems very
wrong.

(%i1) f () := mac () ::= 'foo$

(%i2) translate (f)$

(%i3) mac ();
(%o3) foo

Is there a reason for doing this?

This does not happen with top-level macro definitions translated with
translate_file.

This behavior seems to go at least as far back as Maxima 5.0 (1994).

Here's what's going on:

(def%tr mdefmacro (form)
(meval form) ;; HMM, THIS HAS A SIDE EFFECT AT THE TIME OF TRANSLATION !!
`($any . (meval ',form)))

Several years ago Robert added the comment.

Cheers,
Kris Katterjohn
Richard Fateman
2017-06-14 02:46:51 UTC
Permalink
Raw Message
There is probably a misconception at the core of this bug report,
which is that people might think that

f(x):=
mac()::='foo,
g(z):= x+z,
.....
;

defines a function f which has two local functions,
mac and g. And that the definition of g refers to the value
of x in the call to f(..).

which is false

and that outside of f() there is no definition of mac().

Which is false. the (macro) is defined the first time that
f is called.

Basically, there is no reason to define nested functions
like this (but see local() declarations).

The problems with translate() result from two sources.
(1) It was originally written by an ambitious MIT undergrad
who valued complexity over correctness.
(2) The semantics are sufficiently complicated, rife with
peculiar corner cases, that thorough testing of translation
has been a mostly-unmet challenge.

A good way of fixing translation is to disable it for any
parts that seem difficult or questionable. The purpose
has been to make numerical computation faster. If
that's not the situation, meval() will not be much
slower, and will be correct.

A second purpose (used by the Macsyma company)
was, I think, to obscure the source code, by distributing
only translated code.

RJF
Post by Kris Katterjohn
Hey everyone,
I've noticed that translating a function that contains a macro
definition causes the macro to actually be defined. This seems very
wrong.
(%i1) f () := mac () ::= 'foo$
(%i2) translate (f)$
(%i3) mac ();
(%o3) foo
Is there a reason for doing this?
This does not happen with top-level macro definitions translated with
translate_file.
This behavior seems to go at least as far back as Maxima 5.0 (1994).
(def%tr mdefmacro (form)
(meval form) ;; HMM, THIS HAS A SIDE EFFECT AT THE TIME OF TRANSLATION !!
`($any . (meval ',form)))
Several years ago Robert added the comment.
Cheers,
Kris Katterjohn
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Maxima-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/maxima-discuss
Kris Katterjohn
2017-06-14 04:49:46 UTC
Permalink
Raw Message
Post by Richard Fateman
There is probably a misconception at the core of this bug report,
which is that people might think that
f(x):=
mac()::='foo,
g(z):= x+z,
.....
;
defines a function f which has two local functions,
mac and g. And that the definition of g refers to the value
of x in the call to f(..).
which is false
and that outside of f() there is no definition of mac().
Which is false. the (macro) is defined the first time that
f is called.
Right, but the problem I'm talking about is that the macro is being
defined when f is *translated*. In the example I gave f is not called.

Cheers,
Kris Katterjohn
Robert Dodier
2017-06-14 05:05:51 UTC
Permalink
Raw Message
Post by Kris Katterjohn
(def%tr mdefmacro (form)
(meval form) ;; HMM, THIS HAS A SIDE EFFECT AT THE TIME OF TRANSLATION !!
`($any . (meval ',form)))
Yeah -- I still don't see a motivation for this. It seems desirable to
have translated code act the same as interpreted code, and this
interferes with that.

OK by me if you nuke it. There is tests/rtest_translator.mac which
contains test cases for the translator -- maybe you can add something to
it.

best,

Robert Dodier
Richard Fateman
2017-06-14 15:41:15 UTC
Permalink
Raw Message
I don't know the usage for def%tr and "$any" but
just removing this def%tr should fix
the problem.

Indeed, executing
:lisp (remprop 'mdefmacro 'translate)
solves it.


RJF
Stavros Macrakis
2017-06-14 17:38:38 UTC
Permalink
Raw Message
I haven't read all the code, but I don't think you can just remove/nuke the
meval.

Maxima is a dynamic language, and macros can be defined at any time. They
can be defined at the top level, or within a function. Defining a local
macro inside a function is possible and perhaps even useful, using local().
Defining a global macro inside a function is also possible, but seems like
a bad idea (like other global side-effects).

In any case, for a macro (local or global) to have the correct effect, it
*must* be defined at the time the macro is expanded, whether that is at
translation time or interpretation time. After all, macros can do anything
they want with their arguments; they don't necessarily evaluate them.

On the other hand, the translate subsystem shouldn't change the global
state of Maxima.

The current translator does all this correctly, except that it does leave
behind embedded macro definitions (changing the global state). That is a
bug, but I think it is a smaller bug than not handling embedded macros at
all.

​ -s​
Post by Robert Dodier
Post by Kris Katterjohn
(def%tr mdefmacro (form)
(meval form) ;; HMM, THIS HAS A SIDE EFFECT AT THE TIME OF TRANSLATION
!!
Post by Kris Katterjohn
`($any . (meval ',form)))
Yeah -- I still don't see a motivation for this. It seems desirable to
have translated code act the same as interpreted code, and this
interferes with that.
OK by me if you nuke it. There is tests/rtest_translator.mac which
contains test cases for the translator -- maybe you can add something to
it.
best,
Robert Dodier
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Maxima-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/maxima-discuss
Stavros Macrakis
2017-06-14 21:21:14 UTC
Permalink
Raw Message
Dynamic macro definitions cause all sorts of problems.

Consider for example:

f(n) := (if n=0 then m()::=5 else m()::=6, m())$

This works fine interpreted, but it seems unreasonable to expect any
compiler to deal with it. And sure enough, translate gets it "wrong".

-s
Post by Stavros Macrakis
I haven't read all the code, but I don't think you can just remove/nuke
the meval.
Maxima is a dynamic language, and macros can be defined at any time. They
can be defined at the top level, or within a function. Defining a local
macro inside a function is possible and perhaps even useful, using local().
Defining a global macro inside a function is also possible, but seems like
a bad idea (like other global side-effects).
In any case, for a macro (local or global) to have the correct effect, it
*must* be defined at the time the macro is expanded, whether that is at
translation time or interpretation time. After all, macros can do anything
they want with their arguments; they don't necessarily evaluate them.
On the other hand, the translate subsystem shouldn't change the global
state of Maxima.
The current translator does all this correctly, except that it does leave
behind embedded macro definitions (changing the global state). That is a
bug, but I think it is a smaller bug than not handling embedded macros at
all.
​ -s​
Post by Kris Katterjohn
Post by Kris Katterjohn
(def%tr mdefmacro (form)
(meval form) ;; HMM, THIS HAS A SIDE EFFECT AT THE TIME OF
TRANSLATION !!
Post by Kris Katterjohn
`($any . (meval ',form)))
Yeah -- I still don't see a motivation for this. It seems desirable to
have translated code act the same as interpreted code, and this
interferes with that.
OK by me if you nuke it. There is tests/rtest_translator.mac which
contains test cases for the translator -- maybe you can add something to
it.
best,
Robert Dodier
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Maxima-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/maxima-discuss
Kris Katterjohn
2017-06-14 22:20:01 UTC
Permalink
Raw Message
Post by Stavros Macrakis
I haven't read all the code, but I don't think you can just remove/nuke the
meval.
There are two mevals; one that is evaluated at the time of translation
(the first line of mdefmacro), and another that gets evaluated when the
function is called (the second line of mdefmacro).
Post by Stavros Macrakis
In any case, for a macro (local or global) to have the correct effect, it
*must* be defined at the time the macro is expanded, whether that is at
translation time or interpretation time. After all, macros can do anything
they want with their arguments; they don't necessarily evaluate them.
Removing the first meval (which is evaluated during translation)
shouldn't affect this, should it? The macro will still be defined when
the function is called because of the other meval that will be in the
function body.

I don't want to break anything, so please do let me know if I'm missing
something.

This is after removing the first meval:

(%i1) f () := mac (x) ::= buildq ([x], ['x, x])$

(%i2) g () := 'foo$

(%i3) h () := mac (g ())$

(%i4) translate (f)$

(%i5) h ();
(%o5) mac(foo)

(%i6) f ()$

(%i7) h ();
(%o7) [g(),foo]

(%i8) translate (g, h)$

(%i9) h ();
(%o9) [g(),foo]

Cheers,
Kris Katterjohn
Stavros Macrakis
2017-06-15 15:06:03 UTC
Permalink
Raw Message
​...
There are two mevals; one that is evaluated at the time of translation
(the first line of mdefmacro), and another that gets evaluated when the
function is called (the second line of mdefmacro).
Post by Stavros Macrakis
In any case, for a macro (local or global) to have the correct effect, it
*must* be defined at the time the macro is expanded, whether that is at
translation time or interpretation time. After all, macros can do
anything
Post by Stavros Macrakis
they want with their arguments; they don't necessarily evaluate them.
Removing the first meval (which is evaluated during translation)
shouldn't affect this, should it? The macro will still be defined when
the function is called because of the other meval that will be in the
function body.
No. This doesn't work. You can't "apply" a macro like a function.
Here is an example of the misbehavior if you remove the meval from def%tr
mdefmacro

(def%tr mdefmacro (form)
;;; let's see what happens when we comment this out
;;; (meval form) ;; HMM, THIS HAS A SIDE EFFECT AT THE TIME OF
TRANSLATION !!
`($any . (meval ',form)))


f() := block([m], local(m), m(ex) ::= funmake("'", [ex]), m(print(3)))$
translate(f)$
warning: m is a bound variable in m(print(3)), but it is used as a
function. <<< first sign of problems
note: instead I'll translate it as: apply(m,[print(3)]) <<< oops, can't
apply a macro
=> [f]

f();
3 <<< printed
3 <<< value

​​
To reproduce this, be sure that there are no old definitions of f or m
hanging around.

-s
Kris Katterjohn
2017-06-15 18:01:09 UTC
Permalink
Raw Message
Post by Stavros Macrakis
No. This doesn't work.
Ah, of course. What you were saying makes perfect sense now. Thanks.

Cheers,
Kris Katterjohn
Robert Dodier
2017-06-16 05:10:37 UTC
Permalink
Raw Message
Post by Kris Katterjohn
Ah, of course. What you were saying makes perfect sense now. Thanks.
Maybe you could adjust the comment to reflect our new understanding, and
maybe even output a message from the translator to alert the user that
translation is having a side effect. Just some thoughts.

best

Robert Dodier
Kris Katterjohn
2017-06-16 20:51:51 UTC
Permalink
Raw Message
Post by Robert Dodier
Post by Kris Katterjohn
Ah, of course. What you were saying makes perfect sense now. Thanks.
Maybe you could adjust the comment to reflect our new understanding, and
maybe even output a message from the translator to alert the user that
translation is having a side effect. Just some thoughts.
I added both a comment and a warning message (I think "warning" is
appropriate) in commit 4664fa. Thanks.

Cheers,
Kris Katterjohn
Gunter Königsmann
2017-06-17 15:13:58 UTC
Permalink
Raw Message
Post by Kris Katterjohn
I added both a comment and a warning message (I think "warning" is
appropriate) in commit 4664fa. Thanks.
Currently the warnings maxima produces slightly differ in their form.
Most of them begin with "warning: ", but not all which for gui frontends
makes it harder than necessary for gui frontends to (for example) to
highlight warnings.

Are there reasons for applying the attached patch that provides a
warning(string) command that works analoguous to error(string) (see the
attached patch)?

(%i1) warning("Test");
Warning: Test
(%o1) false
(%i3) warning("test1")$warning("test2")$
Warning: test1
Warning: test2

Kind regards,

Gunter.
Robert Dodier
2017-06-18 07:11:28 UTC
Permalink
Raw Message
Post by Gunter Königsmann
Are there reasons for applying the attached patch that provides a
warning(string) command that works analoguous to error(string) (see the
attached patch)?
I dunno about providing a new function to specifically print a warning
message. Any calls to MTELL which are warnings but don't actually say
"warning:" would have to be changed to call MWARNING instead, but they
can equally well be fixed by just putting "warning:" into the message
string.

best,

Robert Dodier
Gunter Königsmann
2017-06-18 08:57:13 UTC
Permalink
Raw Message
Post by Robert Dodier
Any calls to MTELL which are warnings but don't actually say
"warning:" would have to be changed to call MWARNING instead, but they
can equally well be fixed by just putting "warning:" into the message
string.
That would be an alternative I could live with.

Are there any arguments against me changing all warnings that already contain the word "warning" to begin with "warning: "?

Kind regards,

Gunter.
--
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.
Robert Dodier
2017-06-18 18:25:29 UTC
Permalink
Raw Message
Post by Gunter Königsmann
Are there any arguments against me changing all warnings that already
contain the word "warning" to begin with "warning: "?
I'm not sure which messages you mean. Can you give some examples?

best,

Robert Dodier
Gunter Königsmann
2017-06-19 04:45:20 UTC
Permalink
Raw Message
Post by Robert Dodier
Post by Gunter Königsmann
Are there any arguments against me changing all warnings that already
contain the word "warning" to begin with "warning: "?
I'm not sure which messages you mean. Can you give some examples?
Perhaps the best way to describe what I'm wanting to do is a patch -
which is attached to this message.

Kind regards,

Gunter.
Gunter Königsmann
2017-06-19 04:56:33 UTC
Permalink
Raw Message
I forgot to mention:

I would see things like
ratprint: replacing 0.5 by 1/2 (0.5)
not as "warning", but as "info".
and therefore would not highlight these strings.

Kind regards,

Gunter.
Robert Dodier
2017-06-21 04:46:11 UTC
Permalink
Raw Message
Post by Gunter Königsmann
I would see things like
ratprint: replacing 0.5 by 1/2 (0.5)
not as "warning", but as "info".
and therefore would not highlight these strings.
Agreed -- these are not warning messages.

best

Robert Dodier
Robert Dodier
2017-06-21 04:58:57 UTC
Permalink
Raw Message
Post by Gunter Königsmann
Perhaps the best way to describe what I'm wanting to do is a patch -
which is attached to this message.
I dunno -- some of these I can agree with, but many I don't. I wouldn't
change "warning" to "Warning". I would change "WARNING" and other
variants to "warning".

I made some effort a few years ago to regularize the messages which are
printed in various places. I changed a lot of them but apparently I
overlooked some too.

best,

Robert Dodier

Loading...