As described in Arthur Jaffe and Frank Quinn's ``Theoretical Mathematics'': Toward a Cultural Synthesis of Mathematics and Theoretical Physics it appears to be mainly a call for a loosening of the bonds of rigor. They suggest the creation of a branch of theoretical (experimental) mathematics akin to theoretical physics, where one produces speculative and intuitive works that will later be made reliable through proof. They are concerned about the slow pace of mathematical developments when all the work must be rigorously developed prior to publication. They argue convincingly that a haphazard introduction of conjectorial mathematics will almost undoubtedly result in chaos.
Their solution to the problems involved in the creation of theoretical (experimental) mathematics comes in two parts. They suggest that
theoretical work should be explicitly acknowledged as theoretical and incomplete; in particular, a major share of credit for the final result must be reserved for the rigorous work that validates it. ([9] p.10)This is meant to ensure that there are incentives for following up and proving the conjectured results.
To guarantee that work in this theoretical mode does not affect the reliability of mathematics in general, they propose a linguistic shift.
Within a paper, standard nomenclature should prevail: in theoretical material, a word like ``conjecture'' should replace ``theorem''; a word like ``predict'' should replace ``show'' or ``construct''; and expressions such as ``motivation'' or ``supporting argument'' should replace ``proof.'' Ideally the title and abstract should contain a word like ``theoretical'', ``speculative'', or ``conjectural''. ([9] p.10)Still, none of the newly suggested nomenclature would be entirely out of place in a current research paper. Speculative comments have always had and will always have a place in mathematics.
This is clearly an exploratory form of mathematics. But is it truly experimental in any but the Baconian sense? The answer will of course lie in its application. If we accept the description at face value, all we have is a lessening of rigor, covered by the introduction of a new linguistic structure. More `mathematics' will be produced but it is not clear that this math will be worth more, or even as much as, the math that would have been done without it.
It is not enough to say that mathematical rigor is strangling mathematical productivity. One needs to argue that by relaxing the strictures temporarily one can achieve more. If we view theoretical (experimental) mathematics as a form of Galilean experimentation then in its idealized form `theoretical' (experimental) mathematics should choose between directions (hypotheses) in mathematics. Like any experimental result the answers will not be conclusive, but they will need to be strong enough to be worth acting on.
Writing in this mode, a good theoretical paper should do more than just sketch arguments and motivations. Such a paper should be an extension of the survey paper, defining not what has been done in the field but what the author feels can be done, should be done and might be done, as well as documenting what is known, where the bottlenecks are, etc. In general, we sympathize with the desire to create a `theoretical' mathematics but without a formal structure and methodology it seems unlikely to have the focus required to succeed as a separate field.
One final comment seems in order here. `Theoretical' mathematics, as practiced today, seems a vital and growing instititution. Mathematicians now routinely include conjectures and insights with their work (a trend that seems to be growing). This has expanded in haphazard fashion to include algorithms, suggested algorithms and even pseudo algorithms. We would distinguish our vision of `experimental' mathematics from `theoretical' mathematics by an emphasis on the constructive/algorithmic side of mathematics. There are well established ways of dealing with conjectures but the rules for algorithms are less well defined. Unlike most conjectures, algorithms if sufficiently efficacious soon find their way into general use.
While there has been much discussion of setting up standardized data bases to run algorithms on, this has proceeded even more haphazardly. Addressing these issues of reliability would be part of the purview of experimental mathematics. Not only would one get a critical evaluation of these algorithms but by reducing the problems to their algorithmic core, one may facilitate the sharing of insights both within and between disciplines. At its most extreme, a researcher from one discipline may not need to understand anything more than the outline of the algorithm to make important connections between fields.