Download
Latest Version |
|
Copyright (C) 2005 Distributed
under a |
What is JOpt?
JOpt is an open source Java wrapper that provides objects like Variable, Constraint, and Term and lets you express your linear or mixed integer programs in a natural manner, while remaining agnostic to the details of the solver backend. JOpt is not a solver. Rather, it requires a solver such as CPlex or the free LPSolve to operate. In the future, we aim to provide backend support for at least one free solver, such as hooks into the GNU Linear Programming Kit, etc. This means that any program written with JOpt will work regardless of the solver actually licensed or downloaded by the end user.
JOpt also provides a round-robin load balancer. If you solve multiple concurrent LP/MIPs, and have access to multiple machines with a solver, JOpt can take advantage of your setup to get a linear speed increase.
Why JOpt?
You do linear or mixed integer programming, but want to think in terms of variables and constraints.
You want to automatically distribute and load balance your problems to one or more solver machines.
What is New?
Since the last release:
-
infeasibility analysis (when using CPLEX); see
MIPInfeasibleException
- you can propose initial values to a MIP,
which may speed up MIP solution (when using CPLEX); see
IMIP.*SolveParam functions
Instructions
JOpt ships as one package (Jar File) that contains both a server-side component (which runs the actual solver) and a client-side component (which you integrate with your program). The server-side component can sit on one or many powerful backend server machines, while the client can be anywhere.
JOpt is easy to set up -- assuming that you have Java, Apache Ant, and your favorite solver set up. Otherwise, it takes a few more steps, outlined here:
Setting up the Server (CPLEX 8.0 or higher users)
Verify that you have the same JDK version running on both the client and the servers. This may not be required, but is probably a good idea. JOpt has been tested and should be compatible with JDK 1.4 and 1.5.
Install Apache Ant 1.6.2 or
higher.
If you are
doing Java programming and you don't use Ant, then you are in for a
treat. Ant is the de facto standard buildfile / makefile replacement
and will make your life easier. See http://ant.apache.org/
You can check that you have a recent version installed in your path
by typing:
%
ant -version
Apache Ant version 1.6.2 compiled on September 28
2004
Install CPLEX.
Special Instructions for those at Harvard:
You may or may not find CPLEX on the shared server machines. If
you don't, you need to get the CPLEX distribution onto your machine
of choice.
- Presuming you have an eecs account, CPLEX can be
found on econcs.eecs.harvard.edu at
/usr/ilog/dist/linux/cplex81_1.linux.tar.gz.
(If you do not have an eecs account, contact jeffsh.)
- After
exploding the tar file, you need to copy the /usr/ilog/ilm/access.ilm
on econcs.eecs.harvard.edu to your own directory.
- If you don't
install CPLEX into /usr/ilog/ilm, then you also need to set your
ILOG_LICENSE_FILE environment variable to point to the access.ilm
file on your machine. (See step 4 of
http://www.eecs.harvard.edu/~jeffsh/_cplexdoc/getstart/html/installation3.html#152115)
If using bash, you would do this by adding these commands to ~/.bash_profile:
if
[ -f ~/.bashrc ]; then
source ~/.bashrc
fi
and this command to your ~/.bashrc:
export
ILOG_LICENSE_FILE=/enter/path/to/your/license/file/here/access.ilm
- You should be able to run (from the cplexdir/ilm directory):
%
./ilmcheck /enter/path/to/your/license/file/here/access.ilm
Checking
license file "access.ilm"
License file "access.ilm"
is correct. the License file is correct.
The JOpt server install is actually just a mkdir jopt and then unzip the jopt zip file into this directory. Jopt comes pre-compiled and is in lib/jopt.jar. (The zip file also contains the logging config file, and example, the JOpt source code, this documentation, and the redistribution license file.)
Above the directory where you installed JOpt, create a symbolic
link to your cplex installation directory. (eg: ln -s
/usr/ilog/cplex81 cplexdir). Unless you change build.xml to teach it
otherwise, JOpt running in server-mode assumes that the directory
structure looks something like:
.\cplexdir
(you made this symbolic link
above)
.\cplexdir\bin\i86_linux2_glibc2.3_gcc3.2 (ships with CPLEX
8.1)
.\jopt (you mkdir'd this above)
.\jopt\jopt.jar (you
unzip'd this above)
You are now done with the server side installation. To test, you should run:
ant runcplex
and you should see:
Buildfile:
build.xml
runcplex:
[java] MAIN SolverServer: Binding server to port: 2000
If you don't want to bind to port 2000, change the line in build.xml:
<property name="port" value="2000"/>
That's it! Now read the section “Setting up the Client” Below
Setting up the Server (LPSolve)
To be written.
Setting up the Server (Load balancing for all solvers)
To be written.
Setting up the Client
If you followed the instructions for setting up the server, you are basically there. The zip file comes packaged with two example files in the root directory, ExampleSimple.java and ExampleSimpleWrapper.java. To test them, in bash, you would type exactly:
%
javac -d . -classpath lib/jopt.jar ExampleSimple.java
% java
-classpath .:lib/jopt.jar
edu.harvard.econcs.jopt.example.ExampleSimple
(Note that it is important that your classpath include both the JOpt directory (. in the above example) and the jopt.jar file) and you should see:
Usage: java edu.harvard.econcs.jopt.example.ExampleSimple ServerHostName ServerPortNumber
For instance, if you are running your server on drdoom.eecs.harvard.edu on port 2000, you would type:
% java -cp .:lib/jopt.jar edu.harvard.econcs.jopt.example.ExampleSimple drdoom 2000
and
you should see:
Vars:
y
x
Constraints:
Constraint {[1.0*y] == 2.0}
Constraint {[1.0*x, -2.0*y] <= 5.0}
Objective Function: Max[
2.0*x + 2.0*y ]
Constraint {[1.0*y] == 2.0}
objective: 22.0
x
is 9.0
y is 2.0
Logging
The file edu/harvard/econcs/util/log.config controls the level of logging (screen output) that you will see while running JOpt, in both server-mode and client-mode. The default settings provide very little screen feedback. Increasing the JOpt debugging output, while helpful for debugging, will cause your program to run slowly.
If you are using CPLEX, note that CPLEX does it's own logging in addition to JOpt's screen output. This can be controlled using Jopt's MIP.setSolveParam(MIP_DISPLAY) and related control variables.
JOpt logging just uses Java 1.4 logging, but you can piggy back your own logging code in your own programming that can use what we've already written. In your file MyClass.java, add the line:
private static Log log = new Log(MyClass.class);
Then, after importing import edu.harvard.econcs.util.Log, you can make calls like:
log.debug(“This is a debug message”);
In a complicated program, picking the right level of message printing will be useful to you, since you can control what level of message gets printed by adding a line to the bottom of file log.config in local directory edu/harvard/econcs/util/ For instance, if MyClass is in package edu.harvard.work, you would add the line:
edu.harvard.work.MyClass = DEBUG
if you wanted to see all messages that were DEBUG severity or worse. Note that this overrides the default value that is listed in .level in that same log.config file.
Possible log levels (from most severe to least severe) are:
ERROR
WARNING
MAIN
INFO
TRACE
DEBUG
TRIVIAL
As a final performance note, if the debug output is in a speed-critical part of your code, it is worthwhile to write your code in this fashion:
if (log.isDebugEnabled())
log.debug("Skipping term: " + term);
FAQ
CPLEX
Q. Help!
A.
Read this FAQ, then go to
https://list.eecs.harvard.edu/mailman/listinfo/jopt-users
Q. When I run, I see the following:
java.lang.UnsatisfiedLinkError:
no cplexXX in java.library.path
java.library.path must point to
the directory containing the CPLEX shared library
try invoking
java with java
-Djava.library.path=...
java.lang.UnsatisfiedLinkError:
CPXopenCPLEX
at ilog.cplex.Cplex.CPXopenCPLEX(Native Method)
at
ilog.cplex.CplexI.init(CplexI.java:3601)
at
ilog.cplex.CplexI.<init>(CplexI.java:287)
at
ilog.cplex.IloCplex.<init>(IloCplex.java:8556)
at
edu.harvard.econcs.jopt.solver.impl.CPlexMIPSolver.solve(CPlexMIPSolver.java:40)
at
(etc)
A. This is a CPLEX installation issue. Make sure that you have installed more than just the static CPLEX library during CPLEX installation. Then, make sure you are running Java with something like: -Djava.library.path=C:\ILOG\cplex81\bin\msvc6
Q. Why is JOpt running slowly when I call solve? (My machine is pegged at 100%.)
A. Do you have debugging on? Debugging eats up a lot of local processor on a solve call for large MIPs, as it spits out the MIP it is trying to solve on the console screen. Look at log.config in your local directory edu/harvard/econcs/util/ and look for the line that starts “.level=”. This should be MAIN.
Q. Why is JOpt running slowly when I call solve? (My machine is not pegged.)
A. Does the server-side have debugging on? Debugging eats up a lot of server processor on a solve call for large MIPs, as it spits out all sorts of progress information to the console screen. Look at log.config in your remote directory edu/harvard/econcs/util/ and look for the line that starts “.level=”. This should be MAIN. If you did not set up the server-side component, ask the person who did to check this.
Q. Could it be serialization time / transfer time that is making my program run slowly?
A. Yes, it is possible, especially if your client and server have a poor communications link. But if you haven't already turned off debugging (see previous questions), try that first. Communicating a MIP problem with 1000's of constraints and variables can usually be measured in ms, or a few seconds.
LPs/MIPs are transferred from your program to the server over Java RMI. This arrangement makes efficient use out of these server machines and doesn't let any one user hog a solver license. Our experience has been that the Java RMI time is small compared to the solve time, and if you are using co-racked blades, the RMI serialization / transfer / deserialization time for both the client to server and server back to CPLEX is actually dominated by the time it takes to load CPLEX with the problem!
Q. I am seeing weird behavior when running the client in Eclipse.
A. We think we fixed these, but Eclipse seems to use a different internal compiler that whatever you have installed, which screws you up if you don't have SerialUID since it can generate different SerialUID.
Q. Why do I get a:
ERROR SolverClient: Can't contact server
java.rmi.UnmarshalException: error unmarshalling return; nested exception is:
java.lang.ClassNotFoundException: edu.harvard.econcs.jopt.solver.server.SolverServer_Stub (no security manager: RMI class loader disabled)
A. This is usually because your JOpt client jar is out of date with the server jar.
Q. What memory things should I know about?
A. In your own launching of Java and in the build.xml file on the server side, you may need to specify a -Xmx parameter to avoid a java.lang.OutOfMemoryError for large problems. Run java -X for more information about this error. The default JOpt Server settings is 1GB.
Q. How do I know that seeding values is working?
When using CPLEX, you should see:
[java] MIP start values provide initial solution.
but you know that the starting values weren't accepted if you see:
[java] Warning: MIP start values ignored because of non-optimal
[java] subproblem optimization.
Q: I get an error like:
Can't read log config stream
Jun 23, 2005 11:48:24 AM edu.harvard.econcs.util.Log info
INFO: Contacting Server for remote solver: econcs.eecs.harvard.edu:2001
A: You need to include Jopt directory in the classpath (so it can find log.config, which lives in the edu.harvard.econcs.util directory)
Q. When running the test program, it seems to
hang when making a client connection.
A. If your server is
multi-homed, this can occur. Sun's view of the world for RMI (used by
JOpt) is that every multi-homed host has a 'most public' IP address.
This is somewhat bogus in real life. One workaround is to edit
build.xml to have a -Djava.rmi.server.hostname=REAL-IP-ADDRESS.
We may
have a cleaner workaround in the future.