Date Created: Tue 05-Feb-2008

Related Document Categories:

Get my WebSphere Application Server course here >> http://www.themiddlewareshop.com/products/


How to create you own Java based Download Manager.

How to create you own Java based HTTP / HTTPS download manager with resume. Download manager source code designed with open source code.

In 2006 I had to architect a custom download manager. After evaluating the use of libcurl libraries (http://sourceforge.net/projects/curl/). The team and I found that the Apache Jakarta Commons HttpClient would be a better solution for an HTTP/S resume capable download manager as libcurl would have to either be ported to Java (big task) or use JNI to hook into a dll or so.

The following lists the downloads and basic setup of an Eclipse Java project...

JDK 5.0 Update 7

Installing the JDK SE (http://java.sun.com/javase/downloads/index.jsp)

The Java SE Development Kit (JDK) includes:
  • the Java Runtime Environment (JRE)
  • command-line development tools, such as compilers and debuggers, that are necessary or useful for developing applets and applications

1. Run the windows installer and install the JDK wherever you wish, by convention Java programmers install the JDK in a c:\JSK folder and add a subfolder for the version


e.g. C:\Java\jdk1.5.0_07


Eclipse SDK 3.2
Installing Eclipse IDE (http://www.eclipse.org/downloads/)


1. Unzip the downloaded file (eclipse-SDK-3.2-win32) to c:\eclipse


2. Create a folder called c:\eclipse_workspaces

This is where you will save your projects and keep then separate from the Eclipse IDE installation




Part 1.


Creating your project in Eclipse


1. Run Eclipse, look for the eclipse.exe in the c:\eclipse folder.




2. Select where you wish to save your project e.g. c:\eclipse_workspaces\download




3. Create a new project




4. Select a Java Project




5. Name project and select location




HttpClient Libraries


6. Install HttpClient Libraries


The requirement was to design a native Java application that could be downloaded from a website as a JAR file and run on Linux or Windows hosts and the
HttpClient would fit perfectly.

Although the java.net package provides basic functionality for accessing resources via HTTP, it doesn't provide the full flexibility or functionality needed by many applications. The Jakarta Commons
HttpClient component seeks to fill this void by providing an efficient, up-to-date, and feature-rich package implementing the client side of the most recent HTTP standards and recommendations. See the Features page for more details on standards compliance and capabilities.

The key for resume lies in dealing with the request headers. The Methods that tell us this is possible is
addRequestHeader.

public void addRequestHeader(
Header header)

Adds the specified request header, NOT overwriting any previous value. Note that header-name matching is case insensitive.


Specified by:

addRequestHeader in interface HttpMethod

Parameters:

header - the header to add to the request




8. Adding the HttpClient Common jars to our project.


Add an exclusion setting for the lib folder as this will be the location of our pre-compiled jar files..







Installing the httpclient commons (commons-httpclient-3.0.1.zip)

http://jakarta.apache.org/commons/httpclient/downloads.html


9. Unzip these to your project lib folder e.g.


C:\eclipse_workspaces\download\downloadmanager\lib\bin


Tip: What I do then is copy the jar file to a lib folder, this way I can refer back to the original downloaded api jar file and it's related docs.


We will be referencing our lib folder in our project. (lib meaning external precompiled libraries)


Our lib files will be found in C:\eclipse_workspaces\download\downloadmanager\lib when we add then to our project as shown in the next step.



10. Add jar files. This can be done two ways:


1. Reference the files externally to the project.

2. Reference the jar files in the project e.g in our project we have the jar files located in the \lib directory.



Add the four files above (Download them all from the Alache Jakarta Commons area previously mentioned.)




Looking at the Package Explorer we get a list of the httpclient and related jar files.





11. Creating a class file to begin our application.





Create a package as using the default package is discouraged. In larger projects using a package is useful to minimise ambiguous method, property, variable names in different projects. If you are an avid c# programmer you will know that packages are similar in nature to .NET Namespaces.




12. Copy in this code in to the ResumeTest.java file


//------------------------------------------------------------------------------------------------------------------------------


package downloadmanager;

import org.apache.commons.httpclient.*;

import org.apache.commons.httpclient.methods.*;

import java.io.*;


public class ResumeTest {


private String url;

private String fileName;


public ResumeTest(String url, String fileName)

{

this.url = url;

this.fileName = fileName;

}


public boolean Download()

{

long localFileSize = 0;

long remoteFileSize = 0;

BufferedInputStream in = null;

RandomAccessFile out = null;

HttpMethod request = null;


try {

File file = new File(fileName);

if (file.exists()) {

localFileSize = file.length();

}


HttpClient client = new HttpClient();

request = new GetMethod(url);

if (localFileSize > 0) {

// server must support partial content for resume

request.addRequestHeader("Range", "bytes=" + localFileSize + "-");

if (client.executeMethod(request) != HttpStatus.SC_PARTIAL_CONTENT) {

return false;

}

} else if (client.executeMethod(request) != HttpStatus.SC_OK) {

// response not ok

return false;

}


Header contentLengthHeader = request.getResponseHeader("content-length");

if (contentLengthHeader == null) {

return false;

}

remoteFileSize = Long.parseLong(contentLengthHeader.getValue());


System.out.println("Local file size is " + localFileSize + " bytes.");

System.out.println("Remote file size is " + remoteFileSize + " bytes.");

System.out.println("Bytes Remaining " + (remoteFileSize - localFileSize) + " bytes.");


in = new BufferedInputStream(request.getResponseBodyAsStream());

out = new RandomAccessFile(file, "rw");

out.seek(localFileSize);


int offset = 0;

int len = 4096;
/*

*
* Len is 4 K is the max we can receive in one go.
* Webserver dictates e.g. in a .NET application running in Windows 2002 server as of 01-August-2003 would be
* dictated by the maxRequestLength setting in the machine.config file or web.config file and is the total amount of
* data that can be sent through HTTP Post to the server.
* The default is 4MB (4096)...and is generally set low so that your server will not
* be overwhelmed by possible DoS attacks.

*
* The test link below will be set at 1460 by the provider of the test link i.e. apache.mirror.positive-internet.com

*/


int bytes = 0;


byte[] block = new byte[len];

if ( (bytes = in.read(block, offset, len)) > -1) {

out.write(block, 0, bytes);

}


System.out.println("Bytes written: " + bytes);


return bytes == remoteFileSize;


} catch (Exception e) {

e.printStackTrace();

return false;

} finally {
try {

if (in != null) in.close();

if (out != null) out.close();

} catch (IOException ioe) {

}

request.releaseConnection();

}

}


public static void main(String[] args)

{

String url = "http://apache.mirror.positive-internet.com/jakarta/commons/httpclient/source/commons-httpclient-3.0.1-src.zip";

String file = "c:\\test.zip";

ResumeTest downloader = new ResumeTest(url, file);

downloader.Download();

}

}



//------------------------------------------------------------------------------------------------------------------------------------


13. Turn of build automatically (if you wish, this allows you to control when you wish to re-build your code)




Build Automatically = Off




Run you project.



If you have not recently saved your project, you may get this dialogue.




13. The program is designed to get 100KB of the file at a time. Each time you run the application it will increase the on disk file by 100k. You will have to run the project several times to get the complete file onto disk as we are resuming the file. The resume is is managed by the headers passes to the web server where the file is located.


Part 2.


In part 2 we will describe the code and explain how to build a proper class model and GUI.


This page was last updated 01-August-2006


Get my WebSphere Application Server course here >> http://www.themiddlewareshop.com/products/


Steve Robinson - IBM Champion 2013

About Me

Steve Robinson has been working in IT for over 20 years and has provided solutions for many large-enterprise corporate companies across the world. Steve specialises in Java and Middleware.

In January 2013, I was awarded the prestigous 'IBM Champion' accolade.


  • Linked In
  • Twitter
  • About Me
  • My Blog
  • Contact Me

Read my books?

IBM WebSphere Application Server 8.0 Administration Guide

IBM WebSphere Application Server 8.0 Administration Guide

WebSphere Application Server 7.0 Administration Guide

WebSphere Application Server 7.0 Administration Guide

WebSphere Categories

Other Categories