Attachment API upload with java.net.http

Dave Oshinsky · ‎08-17-2023

Does anyone have a working sample of Java code using java.net.http to upload an attachment for URI "/api/now/attachment/upload"? I've tried various combinations of multipart uploads, all ending up with:

{"error":{"message":"Invalid content-type. Supported request media types for this service are: [multipart/form-data]","detail":null},"status":"failure"}

There appear to be similar (supposedly) working code samples with Apache HTTP, such as:

https://www.servicenow.com/community/developer-forum/attachment-upload-rest-api-java/m-p/1345435

Rather than ditch java.net.http and switch back to Apache HTTPComponents, I thought I'd ask here first. Among other things, java.net.http is missing a multipart publisher (e.g., see Methanol at https://github.com/mizosoft/methanol/blob/master/USER_GUIDE.md), as well as a list of HTTP status codes.

thomaskennedy · ‎08-17-2023

It's not clear if your endpoint limits you to multipart. I don't have a working sample of an upload using that.

This jdk6 sample uploads an attachment using java.net.HttpUrlConnection. All this code had to do was run once, so I was not much concerned with reuse. I could not use anything newer than jdk6.

	public TransmissionResult Post(String url, Map<String,String> values) throws NotesException {
		StringBuilder sb = new StringBuilder();
		
		try {		
			URL urlObj = new URL(url);
			HttpURLConnection connection = (HttpURLConnection) urlObj.openConnection();
			connection.setRequestMethod("POST");
			connection.setRequestProperty("Content-Type", "application/xml");
			connection.setRequestProperty("Accept", "application/json");
			connection.setDoOutput(true);
			
			sb.setLength(0);
			sb.append(this._service_settings.getUsername()).append(":").append(this._service_settings.getPassword());
			String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(sb.toString().getBytes());
			    
			connection.setRequestProperty ("Authorization", basicAuth);
			
			sb.setLength(0);
			sb.append(XML_REQUEST_START).append(XML_ENTRY_START);
			Set<String> keys = values.keySet();
			Iterator<String> iter = keys.iterator();
			while(iter.hasNext()) {
				String column = iter.next();
				sb.append("<").append(column).append(">");
				sb.append(CDATA_ESC_START);
				sb.append( values.get(column) );
				sb.append(CDATA_ESC_END);
				sb.append("</").append(column).append(">");
			}
			sb.append(XML_ENTRY_END).append(XML_REQUEST_END);
				
			OutputStream os = connection.getOutputStream(); 
			byte[] input = sb.toString().getBytes();
			os.write(input, 0, input.length);			
				
			int responseCode = connection.getResponseCode();
			
			BufferedReader inputReader = new BufferedReader( new InputStreamReader(connection.getInputStream()) );
			String inputLine;
			sb.setLength(0);
			while ((inputLine = inputReader.readLine()) != null) {
				sb.append(inputLine);
			}
			inputReader.close();
			String response = sb.toString();
			    
			// We expect a 201 here
			switch(responseCode) {
			case 201:
				// return the sys_id
			    	
			    // 1. Get the index of '"sys_id":'
				int index = response.indexOf("\"sys_id\":");
				         
				// 2. Get the index of the first quote after that
				index = response.indexOf("\"", index + "\"sys_id\":".length());
					    
				// 3. Get the next 32 chars. This is the sys_id.
				String sys_id = response.substring(index+1, index + 33);
			    	
				return new TransmissionResult(TransmissionStatus.INSERTED, sys_id, "OK");
			default:
				return new TransmissionResult(TransmissionStatus.FAILED, response);
			}
			
		} catch(IOException ioe) {
			String stackTrace = Util.ExceptionToString(ioe);
			return new TransmissionResult(TransmissionStatus.FAILED, stackTrace);
		} catch(Exception e) {
			String stackTrace = Util.ExceptionToString(e);
			return new TransmissionResult(TransmissionStatus.FAILED, stackTrace);
		} 
		
	}
	
	public TransmissionResult PostAttachment(String url, byte[] data) throws NotesException {
		StringBuilder sb = new StringBuilder();
		
		try {
			
			URL urlObj = new URL(url);
			HttpURLConnection connection = (HttpURLConnection) urlObj.openConnection();
			connection.setRequestMethod("POST");
			connection.setRequestProperty("Content-Type", "*/*");
			connection.setRequestProperty("Accept", "application/json");
			connection.setDoOutput(true);
			    
			sb.setLength(0);
			sb.append(this._service_settings.getUsername()).append(":").append(this._service_settings.getPassword());
			String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(sb.toString().getBytes());
			sb.setLength(0);
			
			connection.setRequestProperty ("Authorization", basicAuth);
				
			OutputStream os = connection.getOutputStream(); 
			os.write(data, 0, data.length);			
				
			int responseCode = connection.getResponseCode();
			   
			BufferedReader inputReader = new BufferedReader( new InputStreamReader(connection.getInputStream()) );
			String inputLine;
			while ((inputLine = inputReader.readLine()) != null) {
				sb.append(inputLine);
			}
			inputReader.close();
			String response = sb.toString();
			    
			// We expect a 201 here
			switch(responseCode) {
			case 201:
				// return the sys_id
			    	
			    // 1. Get the index of '"sys_id":'
				int index = response.indexOf("\"sys_id\":");
				         
				// 2. Get the index of the first quote after that
				index = response.indexOf("\"", index + "\"sys_id\":".length());
					    
				// 3. Get the next 32 chars. This is the sys_id.
				String sys_id = response.substring(index+1, index + 33);
			    	
				return new TransmissionResult(TransmissionStatus.INSERTED, sys_id, "OK");
			default:
				return new TransmissionResult(TransmissionStatus.FAILED, response);
			}
			
		} catch(IOException ioe) {
			String stackTrace = Util.ExceptionToString(ioe);
			return new TransmissionResult(TransmissionStatus.FAILED, stackTrace);
		} catch(Exception e) {
			String stackTrace = Util.ExceptionToString(e);
			return new TransmissionResult(TransmissionStatus.FAILED, stackTrace);
		} 
	
	}

Dave Oshinsky · ‎08-17-2023

I'm using a Utah PDI in which I have changed no settings related to Attachment API. How would I know whether I'm "limited to multipart", whatever that means?

My latest experiments are using Apache HttpMime (4.5.14) with java.net.http.HttpClient under Java 17. The code is still failing as shown in the TODO. All of the MIME parts are set to multipart/form-data ContentType, but I've tried other ContentType's shown in the commented out code /* whatever */.

public HttpResponse<String> postAttachmentFile(String suffix, String tableName, String tableSysId, Path inputFile) throws URISyntaxException, IOException, InterruptedException {
// TODO: fails with {"error":{"message":"Invalid file type: multipart/form-data","detail":null},"status":"failure"}
final String multipartStr = "multipart/form-data";
ContentType multipart = ContentType.MULTIPART_FORM_DATA; // ContentType.create(multipartStr, StandardCharsets.UTF_8);
MultipartEntityBuilder builder = MultipartEntityBuilder.create()
.addPart("table_name",
new StringBody(tableName, multipart /*ContentType.DEFAULT_TEXT*/))
.addPart("table_sys_id",
new StringBody(tableSysId, multipart /*ContentType.DEFAULT_TEXT*/))
.addBinaryBody("uploadFile", inputFile.toFile(), multipart /*ContentType.DEFAULT_BINARY*/,
inputFile.getFileName().toString());
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
HttpEntity httpEntity = builder.build();

// reading the entire thing into a byte array may not be scalable, when the attachment is extremely large.
// however, the POST() call below requires an InputStream supplier with ofInputStream(),
// since the input rewinds to the beginning of data (verified with InputStreamSupplyOnce).
// we supply a byte array which avoids rewind issues.
byte[] data = Utils.StreamToByteArray(httpEntity.getContent());

final String uriString = baseURI + suffix;
final String httpContentType = httpEntity.getContentType().toString();
HttpRequest.Builder b = HttpRequest.newBuilder().POST(HttpRequest.BodyPublishers.ofByteArray(data))
.header("accept", "application/json")
// The Content-Type header is important, don't forget to set it.
.header("Content-Type", httpContentType)
//.header("Accept-Encoding", "gzip, deflate, br")
.uri(new URI(uriString));
HttpRequest request = b.build();
return httpClient.send(request, HttpResponse.BodyHandlers.ofString());
}

This now looks a lot more like code posted here, but my code is mixing Apache HttpMime for the MIME HTTP entity with Java 11+ java.net.http.HttpClient to run the actual POST:

https://www.servicenow.com/community/developer-forum/attachment-upload-rest-api-java/m-p/1345435

Dave Oshinsky · ‎08-18-2023

I was able to get the "file" (instead of "upload") Attachment API POST to work using code below. It only works if the inputFile data contents actually matches that indicated by the extension - i.e., a *.jpg upload works when the file actually is a jpg, but fails when I rename a text file to whatever.jpg. The question remains open as to what is wrong with the above "upload" code.

public HttpResponse<String> postAttachmentFile(String tableName, String tableSysId, Path inputFile) throws URISyntaxException, IOException, InterruptedException {
StringBuffer sb = new StringBuffer();
sb.append(baseURI);
sb.append("/api/now/attachment/file"); // all earlier experiments failed with "/api/now/attachment/upload"
sb.append("?table_name=");
sb.append(tableName);
sb.append("&table_sys_id=");
sb.append(tableSysId);
sb.append("&file_name=");
sb.append(inputFile.getFileName().toString());
final String uriString = sb.toString();
final String contentType = Files.probeContentType(inputFile);
System.out.println("URI:" + uriString);

try (FileInputStream fis = new FileInputStream(inputFile.toFile())) {
// POST rewinds the InputStream, hence we convert to byte array.
byte[] data = Utils.StreamToByteArray(fis);
HttpRequest.Builder b = HttpRequest.newBuilder().POST(HttpRequest.BodyPublishers.ofByteArray(data))
.header("accept", "application/json")
.header("Content-Type", contentType)
.uri(new URI(uriString));
HttpRequest request = b.build();
return httpClient.send(request, HttpResponse.BodyHandlers.ofString());
}
}