Thanks for your patience. I've just gotten around to looking this over but when I review your fork, it doesn't appear that you've made any changes in your branch.

In general though, it looks like the integration is really straight forward. I'd prefer you to submit a PR for this, and basing it on something like the Azure Application Insights Agent should give you a good head start.

I pushed the git local changes.
Added:

   new file:   config/elastic_apm_agent.yml
   new file:   lib/java_buildpack/framework/elastic_apm_agent.rb

Basically followed the pattern for New Relic.
Thanks and sorry for failing at basic git commit/push. :(

This looks close enough to submit as a formal PR for review. There are some changes we need to make, but nothing major.

What do I need to do for this to get merged?
Do I need to do anything?
I'm going to test this using my custom branch?

I merged my remote branch into the master, updated the docs, and some other items that seems to be required to get the new ElasticApmAgent code and yml configuration working.
Now I'm getting a error in the release section.
Further I turned on the debugging per this document:
https://github.com/cloudfoundry/java-buildpack/blob/master/docs/debugging-the-buildpack.md
Now I'm getting a crazy amount of logging.
It doesn't look like
END OF detailed log: the main error seems like;
Failed to build droplet release: buildpack's release output invalid: yaml: line 1: mapping values are not allowed in this context
Exit status 224

       Verify return code: 0 (ok)
   [DownloadCache]                  DEBUG Adding If-None-Match: 
   [DownloadCache]                  DEBUG Adding If-Modified-Since: 
   [DownloadCache]                  DEBUG Request: /openjdk/trusty/x86_64/index.yml, {"accept-encoding"=>["gzip;q=1.0,deflate;q=0.6,identity;q=0.3"], "accept"=>["*/*"], "user-agent"=>["Ruby"]}
   [DownloadCache]                  DEBUG Response headers: {"content-type"=>["text/x-yaml"], "content-length"=>["6227"], "connection"=>["keep-alive"], "date"=>["Tue, 22 Jan 2019 18:34:54 GMT"], "last-modified"=>["Tue, 22 Jan 2019 18:34:23 GMT"], "etag"=>["\"61babcae65885e33a81b9078176785d5\""], "x-amz-version-id"=>["oQHGk1CwR3mNyQMDuR6_DuJKNdm2PbIi"], "accept-ranges"=>["bytes"], "server"=>["AmazonS3"], "age"=>["3722369"], "x-cache"=>["Hit from cloudfront"], "via"=>["1.1 b27a02320ae6ae78c5c25db5c9ccb9dc.cloudfront.net (CloudFront)"], "x-amz-cf-id"=>["xRlMS_-Z0kqEG_zuTj2NRhtlrybv38Fl4u2ZL6DK2IpRdT4tYYOrFw=="]}
   [DownloadCache]                  DEBUG Response status: 200
   [DownloadCache]                  DEBUG Persisting Etag: "61babcae65885e33a81b9078176785d5"
   [DownloadCache]                  DEBUG Persisting Last-Modified: Tue, 22 Jan 2019 18:34:23 GMT
   [DownloadCache]                  DEBUG Persisting content to /tmp/06e39cddcbd90ad303ff803a925d6614266c3d55f875c5f1ca2c4180056d5b04.cached
   [DownloadCache]                  DEBUG Validated content size 6227 is 6227
   [RepositoryIndex]                DEBUG {"1.7.0_06"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_06.tar.gz", "1.7.0_07"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_07.tar.gz", "1.7.0_08"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_08.tar.gz", "1.7.0_09"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_09.tar.gz", "1.7.0_10"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_10.tar.gz", "1.7.0_11"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_11.tar.gz", "1.7.0_12"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_12.tar.gz", "1.7.0_13"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_13.tar.gz", "1.7.0_14"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_14.tar.gz", "1.7.0_15"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_15.tar.gz", "1.7.0_17"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_17.tar.gz", "1.7.0_21"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_21.tar.gz", "1.7.0_25"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_25.tar.gz", "1.7.0_40"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_40.tar.gz", "1.7.0_45"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_45.tar.gz", "1.7.0_51"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_51.tar.gz", "1.7.0_55"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_55.tar.gz", "1.7.0_60"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_60.tar.gz", "1.7.0_65"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_65.tar.gz", "1.7.0_71"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_71.tar.gz", "1.7.0_75"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_75.tar.gz", "1.7.0_79"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.7.0_79.tar.gz", "1.8.0_00"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0.tar.gz", "1.8.0_05"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_05.tar.gz", "1.8.0_101"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_101.tar.gz", "1.8.0_111"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_111.tar.gz", "1.8.0_11"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_11.tar.gz", "1.8.0_121"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_121.tar.gz", "1.8.0_131"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_131.tar.gz", "1.8.0_141"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_141.tar.gz", "1.8.0_144"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_144.tar.gz", "1.8.0_152"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_152.tar.gz", "1.8.0_161"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_161.tar.gz", "1.8.0_162"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_162.tar.gz", "1.8.0_172"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_172.tar.gz", "1.8.0_181"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_181.tar.gz", "1.8.0_191"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_191.tar.gz", "1.8.0_192"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_192.tar.gz", "1.8.0_201"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_201.tar.gz", "1.8.0_202"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_202.tar.gz", "1.8.0_20"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_20.tar.gz", "1.8.0_25"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_25.tar.gz", "1.8.0_31"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_31.tar.gz", "1.8.0_40"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_40.tar.gz", "1.8.0_45"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_45.tar.gz", "1.8.0_51"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_51.tar.gz", "1.8.0_60"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_60.tar.gz", "1.8.0_65"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_65.tar.gz", "1.8.0_71"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_71.tar.gz", "1.8.0_73"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_73.tar.gz", "1.8.0_77"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_77.tar.gz", "1.8.0_91-unlimited-crypto"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_91-unlimited-crypto.tar.gz", "1.8.0_91"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_91.tar.gz", "1.8.0_RC1"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_RC1.tar.gz", "10.0.0_46"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-10.0.0_46.tar.gz", "10.0.1_10"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-10.0.1_10.tar.gz", "10.0.2_13"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-10.0.2_13.tar.gz", "11.0.0_28"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-11.0.0_28.tar.gz", "11.0.1_13"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-11.0.1_13.tar.gz", "11.0.2_07"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-11.0.2_07.tar.gz", "11.0.2_09"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-11.0.2_09.tar.gz", "9.0.0_181"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-9.0.0_181.tar.gz", "9.0.1_11"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-9.0.1_11.tar.gz", "9.0.4_11"=>"https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-9.0.4_11.tar.gz"}
   [ConfigurationUtils]             DEBUG Configuration from /tmp/buildpackdownloads/a457f9495309d44a788ea9f361d85ebf/config/java_main.yml: {"java_main_class"=>nil, "arguments"=>nil}
   [Buildpack]                      DEBUG Release Payload:
   ---
   addons: []
   config_vars: {}
   default_process_types:
     web: 'JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.16.0_RELEASE=printHeapHistogram=1
       -XX:ActiveProcessorCount=$(nproc) -Djava.io.tmpdir=$TMPDIR -Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext
       -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security $JAVA_OPTS"
       && CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.13.0_RELEASE
       -totMemory=$MEMORY_LIMIT -loadedClasses=23908 -poolType=metaspace -stackThreads=250
       -vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY
       && JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && MALLOC_ARENA_MAX=2 SERVER_PORT=$PORT
       eval exec $PWD/.java-buildpack/open_jdk_jre/bin/java $JAVA_OPTS -cp $PWD/. org.springframework.boot.loader.JarLauncher'
     task: 'JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.16.0_RELEASE=printHeapHistogram=1
       -XX:ActiveProcessorCount=$(nproc) -Djava.io.tmpdir=$TMPDIR -Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext
       -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security $JAVA_OPTS"
       && CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.13.0_RELEASE
       -totMemory=$MEMORY_LIMIT -loadedClasses=23908 -poolType=metaspace -stackThreads=250
       -vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY
       && JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && MALLOC_ARENA_MAX=2 SERVER_PORT=$PORT
       eval exec $PWD/.java-buildpack/open_jdk_jre/bin/java $JAVA_OPTS -cp $PWD/. org.springframework.boot.loader.JarLauncher'
   Failed to build droplet release: buildpack's release output invalid: yaml: line 1: mapping values are not allowed in this context
   Exit status 224
   Cell 53d3700d-cd4d-46dd-9e38-67da5100086b stopping instance 8237ff17-026b-45c2-b21c-a62d5e95038d
   Cell 53d3700d-cd4d-46dd-9e38-67da5100086b destroying container for instance 8237ff17-026b-45c2-b21c-a62d5e95038d
   Cell 53d3700d-cd4d-46dd-9e38-67da5100086b successfully destroyed container for instance 8237ff17-026b-45c2-b21c-a62d5e95038d
FAILED

I added debugging print statements to the code so i could see what was happening:
CONFIG makes it there:
[Droplet] DEBUG Sandbox root: /tmp/app/.java-buildpack/dynatrace_one_agent [Buildpack] DEBUG Instantiating JavaBuildpack::Framework::ElasticApmAgent [Buildpack] DEBUG Successfully required JavaBuildpack::Framework::ElasticApmAgent [ConfigurationUtils] DEBUG Configuration from /tmp/buildpackdownloads/a457f9495309d44a788ea9f361d85ebf/config/elastic_apm_agent.yml: {"version"=>"1.1.0", "repository_root"=>"https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent/", "repository_download"=>"https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent/1.1.0/elastic-apm-agent-1.1.0.jar"} [Droplet] DEBUG Droplet root: /tmp/app [Droplet] DEBUG Buildpack root: /tmp/app/.java-buildpack [Droplet] DEBUG Sandbox root: /tmp/app/.java-buildpack/elastic_apm_agent [Buildpack] DEBUG Instantiating JavaBuildpack::Framework::GoogleStackdriverDebugger
supports? - ElasticApmAgent < JavaBuildpack::Component::VersionedDependencyComponent
shows up in the log, but the debug print statements for compile and release do not.
See code here:
https://github.com/kmacpher67/java-buildpack/blob/master/lib/java_buildpack/framework/elastic_apm_agent.rb#L29

ahhh release step calls configuration.
ahha part 2: keys are case sensitive.

cf update-user-provided-service elasticapm -p 'server_urls,application_packages'
I had previously updated the service using the ALL Upper Case key names, where were the variable names in the .rb code. :( are not cool: SERVER_URLSSERVER_URLS,APPLICATION_PACKAGES' bad error handling in that code base.
NOTE TO FUTURE:

Ok, I'm getting this problem now:

[ConfigurationUtils]             DEBUG Configuration from /tmp/buildpackdownloads/a457f9495309d44a788ea9f361d85ebf/config/elastic_apm_agent.yml: {"version"=>"1.1.0", "repository_root"=>"https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent/", "repository_download"=>"https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent/1.1.0/elastic-apm-agent-1.1.0.jar"}

   [RepositoryIndex]                DEBUG https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent/ expanded to https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent

   [DownloadCache]                  DEBUG Request: /maven2/co/elastic/apm/elastic-apm-agent/index.yml, {"accept-encoding"=>["gzip;q=1.0,deflate;q=0.6,identity;q=0.3"], "accept"=>["*/*"], "user-agent"=>["Ruby"]}
   [DownloadCache]                  DEBUG Response headers: {"last-modified"=>["Wed, 10 Aug 2016 15:08:35 GMT"], "etag"=>["\"1fb066da6a67c7c02962f59b4b8cd1ee\""], "x-amz-error-code"=>["NoSuchKey"], "x-amz-error-message"=>["The specified key does not exist."], "x-amz-error-detail-key"=>["maven2/co/elastic/apm/elastic-apm-agent/index.yml"], "content-type"=>["text/html"], "via"=>["1.1 varnish", "1.1 varnish"], "content-length"=>["554"], "accept-ranges"=>["bytes"], "date"=>["Thu, 07 Mar 2019 16:18:19 GMT"], "age"=>["0"], "connection"=>["keep-alive"], "x-served-by"=>["cache-iad2126-IAD, cache-bwi5024-BWI"], "x-cache"=>["MISS, MISS"], "x-cache-hits"=>["0, 0"], "x-timer"=>["S1551975499.474858,VS0,VE25"]}
   [DownloadCache]                  DEBUG Response status: 404
   [DownloadCache]                  WARN  Request failure 1, retrying after 5s.  Failure: 404 Not Found

It's trying to find a /maven2/co/elastic/apm/elastic-apm-agent/index.yml file which doesn't exist.
Crap the maven repo doesn't support index.yml
http://central.maven.org/maven2/co/elastic/apm/elastic-apm-agent/
What's the next steps to work around this?

After an extensive amount of hacking (invalid uris, bad def required detect blocks etc) and revisions:
eventually going with download_jar(@version, @uri, @jar_name )
i get this error, likely due to extensive hacking where I should not have done.

   addons: []
   config_vars: {}
   default_process_types:
     web: 'JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.16.0_RELEASE=printHeapHistogram=1
       -XX:ActiveProcessorCount=$(nproc) -Djava.io.tmpdir=$TMPDIR -Delastic.apm.log_file_name=STDOUT
       -Delastic.apm.server_urls=http://elk5-sts1-a.sapphirepri.com -Delastic.apm.application_packages=kenmac
       was here! -Delastic.apm.elastic.apm.service_name=sts1-TherapyReportTemplates-Server-kentest
       -javaagent:$PWD/.java-buildpack/elastic_apm_agent/elastic-apm-agent.jar -Delkapmagent.home=$PWD/.java-buildpack/elastic_apm_agent
       -Delastic.apm.application_packages.enable.java.8=true -Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext
       -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security $JAVA_OPTS"
       && CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.13.0_RELEASE
       -totMemory=$MEMORY_LIMIT -loadedClasses=25019 -poolType=metaspace -stackThreads=250
       -vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY
       && JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && MALLOC_ARENA_MAX=2 SERVER_PORT=$PORT
       eval exec $PWD/.java-buildpack/open_jdk_jre/bin/java $JAVA_OPTS -cp $PWD/. org.springframework.boot.loader.JarLauncher'
     task: 'JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.16.0_RELEASE=printHeapHistogram=1
       -XX:ActiveProcessorCount=$(nproc) -Djava.io.tmpdir=$TMPDIR -Delastic.apm.log_file_name=STDOUT
       -Delastic.apm.server_urls=http://elk5-sts1-a.sapphirepri.com -Delastic.apm.application_packages=kenmac
       was here! -Delastic.apm.elastic.apm.service_name=sts1-TherapyReportTemplates-Server-kentest
       -javaagent:$PWD/.java-buildpack/elastic_apm_agent/elastic-apm-agent.jar -Delkapmagent.home=$PWD/.java-buildpack/elastic_apm_agent
       -Delastic.apm.application_packages.enable.java.8=true -Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext
       -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security $JAVA_OPTS"
       && CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.13.0_RELEASE
       -totMemory=$MEMORY_LIMIT -loadedClasses=25019 -poolType=metaspace -stackThreads=250
       -vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY
       && JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && MALLOC_ARENA_MAX=2 SERVER_PORT=$PORT
       eval exec $PWD/.java-buildpack/open_jdk_jre/bin/java $JAVA_OPTS -cp $PWD/. org.springframework.boot.loader.JarLauncher'
   Failed to build droplet release: buildpack's release output invalid: yaml: line 1: mapping values are not allowed in this context
   Exit status 224

@nebhale

Is it a requirement that index.yml in the maven repo under
https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent?

What is the best practice for the location of the index.yml ? Are there other common locations (github etc)

In general people simply host binaries anywhere they want. So if you look at config/add_dynamics_agent.yml they just host is somewhere on their own domain.

This is our preferred option (allow the projects to maintain the repo and update as required), but if that's too much of a burden, we have the java-buildpack-dependency-builder project where we replicate dependencies into a cache hosted by the Cloud Foundry Foundation. There are some limitations for this around licensing and download availability but we can explore them if necessary.

Thanks....

Sorry to be dense / So to be clear:

a) We can host the binaries where we want, for us maven repo

b) we can host the index.yml which points to those binaries at

https://github.com/cloudfoundry/java-buildpack/tree/master/config/add_elastic_agent.yml

c) And we would open a pull request to do that. we would become CF Contributors by signing the CLA

Correct?

Is there an order to adding the buildpack + index.yml does that all need to happen as part of the same PR

No problem at all.

1. Binaries can be hosted anywhere you want.
2. The index.yml is used for indirection to those binaries. So there needs to be some URI U, such that U/index.yml exists and contains a YAML hash of versions to download URIs. So for example you could host https://raw.githubusercontent.com/elastic/apm-agent-java/master/index.yml with links to Maven Central and your config/elastic_apm_agent.yml would include repository_root: https://raw.githubusercontent.com/elastic/apm-agent-java/master.

For testing purposes I've done that "recursively" with the same code repo hosting the jar files to get the repo to build, but I'm still getting release errors.

@bvader
https://github.com/kmacpher67/java-buildpack/tree/master/elastic-repo
If you copy this sub-directory over to an elastic repo as Ben suggests, then you could put your binaries there, I could update this repo to point at those raw files.

@nebhale i'm still getting horrible misc errors:
``` [ConfigurationUtils] DEBUG Configuration from /tmp/buildpackdownloads/a457f9495309d44a788ea9f361d85ebf/config/java_main.yml: {"java_main_class"=>nil, "arguments"=>nil}
[Buildpack] DEBUG Release Payload:

addons: []
config_vars: {}
default_process_types:
web: 'JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.16.0_RELEASE=printHeapHistogram=1
-XX:ActiveProcessorCount=$(nproc) -Djava.io.tmpdir=$TMPDIR -javaagent:$PWD/.java-buildpack/elastic_apm_agent/elastic_apm_agent-1.4.0.jar
-Delkapmagent.home=$PWD/.java-buildpack/elastic_apm_agent -Delastic.apm.application_packages.enable.java.8=true
-Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext
-Djava.security.properties=$PWD/.java-buildpack/java_security/java.security $JAVA_OPTS"
&& CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.13.0_RELEASE
-totMemory=$MEMORY_LIMIT -loadedClasses=25019 -poolType=metaspace -stackThreads=250
-vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY
&& JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && MALLOC_ARENA_MAX=2 SERVER_PORT=$PORT
eval exec $PWD/.java-buildpack/open_jdk_jre/bin/java $JAVA_OPTS -cp $PWD/. org.springframework.boot.loader.JarLauncher'
task: 'JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.16.0_RELEASE=printHeapHistogram=1
-XX:ActiveProcessorCount=$(nproc) -Djava.io.tmpdir=$TMPDIR -javaagent:$PWD/.java-buildpack/elastic_apm_agent/elastic_apm_agent-1.4.0.jar
-Delkapmagent.home=$PWD/.java-buildpack/elastic_apm_agent -Delastic.apm.application_packages.enable.java.8=true
-Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext
-Djava.security.properties=$PWD/.java-buildpack/java_security/java.security $JAVA_OPTS"
&& CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.13.0_RELEASE
-totMemory=$MEMORY_LIMIT -loadedClasses=25019 -poolType=metaspace -stackThreads=250
-vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY
&& JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && MALLOC_ARENA_MAX=2 SERVER_PORT=$PORT
eval exec $PWD/.java-buildpack/open_jdk_jre/bin/java $JAVA_OPTS -cp $PWD/. org.springframework.boot.loader.JarLauncher'
Failed to build droplet release: buildpack's release output invalid: yaml: unmarshal errors:
line 1: cannot unmarshal !!str Downloa... into buildpackrunner.Release
Exit status 224```

Ben should I just resubmit what I have do you need me to get these misc release errors resolved.
I need to spec too. I didn't do that yet.
I'm using the master branch now, I couldn't figure out how to get the branch to load in via the manifest.
So i just merged back to my repos master.
https://github.com/kmacpher67/java-buildpack
Let me know what to do.

This is almost certainly because you're outputting something to the console during the release phase. stdout is reserved during that phase in order to communicate the process configuration payload.

Once you simplify the code to utilize all of our downloading behavior and match the other APM integration designs, I think you'll find this problem disappear.

@kmacpher67 @nebhale

@bvader
https://github.com/kmacpher67/java-buildpack/tree/master/elastic-repo
If you copy this sub-directory over to an elastic repo as Ben suggests, then you could put your binaries there, I could update this repo to point at those raw files.

I am working with our tech lead for the Elastic Java APM agent. We are working on the configuration options / decisions. I suspect we will be leaving the binaries in the the maven repo as all our other documentation / dependencies rely on that.

You can see this discussion here.
https://github.com/elastic/apm-agent-java/issues/523#issuecomment-472604463

@kmacpher67 Appreciate your effort on this!

I think we would want to update the docs for the Elastic APM agent, perhaps they should not be JRebel specific. Maybe I am missing something but shouldn't this just be generic buildpack docs for Elastic APM?

I am referring to:
https://github.com/kmacpher67/java-buildpack/blob/master/docs/framework-elastic_apm_agent.md

Also it looks like perhaps there are a number of additional parameters that can / should be supported. maybe after we get this initial PR in the Elastic team can take a look and update.

https://github.com/kmacpher67/java-buildpack/blob/master/lib/java_buildpack/framework/elastic_apm_agent.rb

Example secret_token is a very important config.
https://www.elastic.co/guide/en/apm/agent/java/current/config-reporter.html#config-secret-token

The Elastic Team is looking forward to contributing.

@nebhale working on CLA, believe you and I have crossed paths, I was the VP Eng. at CoreLogic where we used PCF / Labs, help build labs in SaMo now at Elastic

@nebhale do you have plans of supporting maven repositories directly? https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent/maven-metadata.xml should have all required information available, just the format is a little different.

For example

maven_repository_root: https://repo1.maven.org/maven2/co/elastic/apm/elastic-apm-agent
maven_artifact: elastic-apm-agent-$version.jar

@bvader
I've updated the documentation for the agent.
Added the service override for secret_token
ttps://github.com/kmacpher67/java-buildpack

There's an optional property file that we add, do you have sample one that I can +add to the repo:
The build pack has a location for that:
https://github.com/kmacpher67/java-buildpack/tree/master/resources

+1 vote for maven XML support, I would have written it but I'm not a ruby programmer and I doubt the cf folks will given other agents have the same situation. It would be ideal if you can add index.yml if not, let me know, I'll check out the aforementioned project for getting the pivotal folks for staging those item.

I doubt the cf folks will given other agents have the same situation

I think this is an argument for adding maven support - that would make it not only easier for Elastic APM, but also for others 🙂

In the meantime, I have added the index.yml to the github repo: https://github.com/elastic/apm-agent-java/pull/524

@kmacpher67

The index.yml can now be found at:
https://raw.githubusercontent.com/elastic/apm-agent-java/master/cloudfoundry/index.yml

1) Thanks for the update on the secret_token

2) With respect to the documentation, I am still pretty confused about all the JRebel references and JRebel detection etc. The Elastic APM agent / buildpack should not have any 3rd party dependencies / references and hyperlinks backs to JRebel. We can provide instructions in the Elastic Java APM docs soon.

I would expect this page to look more like the New Relic Page. Or perhaps I am completely missing something and this somehow tightly coupled with JRebel.

https://github.com/cloudfoundry/java-buildpack/blob/master/docs/framework-new_relic_agent.md

I am happy to get in and help through your fork or as soon as we merge. Perhaps just take out all the JRebel Reference to start.

example

Elastic APM Agent Framework

The Elastic APM Agent Framework causes causes an application to be automatically configured to work with a bound Elastic APM Service.

Detection Criterion | Existence of a single bound New Relic service.Existence of a New Relic service is defined as the VCAP_SERVICES payload containing a service who's name, label or tag has elastic-apm as a substring.
-- | --
Tags | elastic-agent=<version>

3) We have an unofficial properties file but all properties have reasonable defaults, lets merge without that I will need to get the engineering team involved etc. and we can add that later unless that is absolutely required at first.

@nebhale Is there a way to mark this as "Beta or Experimental" at first? Or does it just go in and we iterate, I am a bit unclear on the process. Elastic will want to iterate on this great work that @kmacpher67 has provided as a foundation.

fix the documentation copy & paste. removed jrebel zero refs.

I'm stilling getting some sort of console logs of Downloa... in the release payload

   Failed to build droplet release: buildpack's release output invalid: yaml: unmarshal errors:
     line 1: cannot unmarshal !!str `Downloa...` into buildpackrunner.Release
   Exit status 224

it looks like it's coming from the following golang wrapper.
https://github.com/cloudfoundry/buildpackapplifecycle/blob/master/buildpackrunner/runner.go#L484
Should I just ask for the Pull Request off of my master repo?
The lastest update works with pointing at the index.yml located at Elastic.

@nebhale Is there a way to mark this as "Beta or Experimental" at first? Or does it just go in and we iterate, I am a bit unclear on the process. Elastic will want to iterate on this great work that

I wrote the spec tests for this , i serious doubt the quality of the tests as I don't have ruby running locally.
i've been hacking and pushing the build pack changes into a cf app space.

Test code:
https://github.com/kmacpher67/java-buildpack/blob/368dc9cc1a08670b49cdbf9d226eaae220ab4b77/spec/java_buildpack/framework/elastic_apm_agent_spec.rb#L80

Ideally once I get this working deploy going to a CF app space, I swing back over to our Elastic 6.4 instance and enable the apm features. See what starts streaming over there.

ohh. `Happy learned to putt!!" actually i was able to look at a diff and realized there were still debugging in the downloadcache.rb that i forgot to remove. lol.

       eval exec $PWD/.java-buildpack/open_jdk_jre/bin/java $JAVA_OPTS -cp $PWD/. org.springframework.boot.loader.JarLauncher'
   Exit status 0
   Uploading droplet, build artifacts cache...
   Uploading droplet...
   Uploading build artifacts cache...
   Uploaded build artifacts cache (52M)
   Uploaded droplet (106.1M)
   Uploading complete
   Cell 3f2ea5c4-31a0-487d-86f9-3020c48e4235 stopping instance 2001f7e3-9fb3-4936-ac28-ef571e1b86e2
   Cell 3f2ea5c4-31a0-487d-86f9-3020c48e4235 destroying container for instance 2001f7e3-9fb3-4936-ac28-ef571e1b86e2
   Cell 3f2ea5c4-31a0-487d-86f9-3020c48e4235 successfully destroyed container for instance 2001f7e3-9fb3-4936-ac28-ef571e1b86e2

Waiting for app to start...
     state     since                  cpu      memory         disk           details
#0   running   2019-03-14T17:39:16Z   296.8%   597.4M of 1G   192.4M of 1G  

It doesn't crash now! Let me update the target pull branch.
Then I'll fix my local cf ELK6.4 install for APM to be installed.
See what/if anything reports back.

@nebhale Do you want me to squash/squanchy the commits? There's my +600 hacking iterations in there as commits.

https://github.com/cloudfoundry/java-buildpack/pull/696

@bvader
it's trying to hit the non-existing port8200 for the apm. I need to set that up now.
Do you have a quick readme for the lazy guy on a quick start setup guide for elastic apm?
```
2019-03-14T13:39:43.03-0400 [APP/PROC/WEB/0] OUT 2019-03-14 17:39:43.037 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type METRICS with this error: connect timed out
2019-03-14T13:39:43.03-0400 [APP/PROC/WEB/0] OUT 2019-03-14 17:39:43.038 [apm-reporter] INFO co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Backing off for 1 seconds (±10%)````

@kmacpher67

You can use this if you are docker comfortable. This will stand up a little Elastic, Kibana, Apm-Server environment

https://gist.github.com/bvader/9665fa7b3bd69457517e41a7c28b4725

Then
TAG=6.6.2 docker-compose -f elastic-apm-compose.yml up

# After the stack is full running please go to, this is how you install / configure the Kibana / Elasticsearch side dashboards etc. 
http://localhost:5601/app/kibana#/home/tutorial/apm?_g=()

Here is a sample java app with instructions for local Java... and everything to test.
https://gist.github.com/bvader/cc9fea7e6f8f709dfa2ad14776eed7fe

If you just want the apm-server then I would just download / install and do minimum config. I would just download the tar.gz and minimum config.

https://www.elastic.co/downloads/apm

You should just need need to configure the output to your elasticsearch instance and the apm-server host

apm-server:
  # Defines the host and port the server is listening on.  use "unix:/path/to.sock" to listen on a unix domain socket.
  host: "10.168.0.3:8200"

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  # Scheme and port can be left out and will be set to the default (http and 9200)
  # In case you specify and additional path, the scheme is required: http://localhost:9200/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:9200
  hosts: ["localhost:9200"]

  # Boolean flag to enable or disable the output module.
  #enabled: true

  # Set gzip compression level.
  #compression_level: 0

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

Then run

https://www.elastic.co/guide/en/apm/server/current/setting-up-and-running.html

./apm-server -e

Also follow the instructions, this is how you upload / configure the Kibana / Elasticsearch side dashboards etc.

# After the stack is full running please go to
http://<kibanahost>:5601/app/kibana#/home/tutorial/apm?_g=()

@kmacpher67 Let me know if you want a little help / collaborate on the APM server perhaps we can jump on a zoom. You can contact me directly at my email...

Oh an apologies if I was confusing on the "Beta or Experimental" at first? " comment I am just learning the CF process and the building the buildpack process, I / We / Elastic / The OSS Community totally appreciate your effort and community spirit.

Next week I will try to pull down your repo if it is not merged and try to install / run on my end.

@nebhale

I think we are ready for you to take a look at the PR when you are ready.

I'm waiting on the Merge Request PULL, but there's a conflict.
Review requested
Review has been requested on this pull request. It is not required to merge.
@nebhale
nebhale was requested for review
This branch has conflicts that must be resolved
Only those with write access to this repository can merge pull requests.
Conflicting files

rakelib/versions_task.rb

I think i need to cherry pick the changes, and move them over to the latest, perhaps do a new FORK based on the latest version.
I've asked on this MR thread, if i should collapse or resubmit.

This should be closed per pull request
https://github.com/cloudfoundry/java-buildpack/pull/696