#350 Gluster: CI job failures due to java IOException
Closed: Fixed 2 years ago by anoopcs. Opened 2 years ago by anoopcs.

I have been noticing the following failure with few jobs, listed towards the end, from Gluster space since Jenkins shutdown:

Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on gluster-ci-slave07 (gluster) in workspace /home/gluster/workspace/gluster_samba-build-rpms-centos7-master
The recommended git tool is: NONE
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from n4-131.cloud.ci.centos.org/172.19.4.131:52470
        at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
        at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
        at hudson.remoting.Channel.call(Channel.java:1001)
        at hudson.FilePath.act(FilePath.java:1158)
        at hudson.FilePath.act(FilePath.java:1147)
        at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:121)
        at hudson.plugins.git.GitSCM.createClient(GitSCM.java:904)
        at hudson.plugins.git.GitSCM.createClient(GitSCM.java:835)
        at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1288)
        at hudson.scm.SCM.checkout(SCM.java:505)
        at hudson.model.AbstractProject.checkout(AbstractProject.java:1204)
        at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:636)
        at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:508)
        at hudson.model.Run.execute(Run.java:1907)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:97)
        at hudson.model.Executor.run(Executor.java:429)
java.lang.NoClassDefFoundError: Could not initialize class com.sun.proxy.$Proxy46
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:739)
    at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:169)
    at hudson.remoting.Channel.export(Channel.java:813)
    at hudson.remoting.Channel.export(Channel.java:776)
    at org.jenkinsci.plugins.gitclient.LegacyCompatibleGitAPIImpl.writeReplace(LegacyCompatibleGitAPIImpl.java:204)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.io.ObjectStreamClass.invokeWriteReplace(ObjectStreamClass.java:1244)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1136)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at hudson.remoting.UserRequest._serialize(UserRequest.java:263)
    at hudson.remoting.UserRequest.serialize(UserRequest.java:272)
    at hudson.remoting.UserRequest.perform(UserRequest.java:222)
    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
    at hudson.remoting.Request$2.run(Request.java:375)
    at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:118)
    at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on JNLP4-connect connection from n4-131.cloud.ci.centos.org/172.19.4.131:52470 failed
    at hudson.remoting.Channel.call(Channel.java:1007)
    at hudson.FilePath.act(FilePath.java:1158)
    at hudson.FilePath.act(FilePath.java:1147)
    at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:121)
    at hudson.plugins.git.GitSCM.createClient(GitSCM.java:904)
    at hudson.plugins.git.GitSCM.createClient(GitSCM.java:835)
    at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1288)
    at hudson.scm.SCM.checkout(SCM.java:505)
    at hudson.model.AbstractProject.checkout(AbstractProject.java:1204)
    at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:636)
    at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:508)
    at hudson.model.Run.execute(Run.java:1907)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    at hudson.model.ResourceController.execute(ResourceController.java:97)
    at hudson.model.Executor.run(Executor.java:429)
Performing Post build task...
Match found for :Building remotely : True
Logical operation result is TRUE
Running script  : # cico-node-done-from-ansible.sh
# A script that releases nodes from a SSID file written by
SSID_FILE=${SSID_FILE:-$WORKSPACE/cico-ssid}

for ssid in $(cat ${SSID_FILE})
do
    cico -q node done $ssid
done

[gluster_samba-build-rpms-centos7-master] $ /bin/sh -xe /tmp/jenkins8935985436784825900.sh
+ SSID_FILE=/home/gluster/workspace/gluster_samba-build-rpms-centos7-master/cico-ssid
++ cat /home/gluster/workspace/gluster_samba-build-rpms-centos7-master/cico-ssid
+ for ssid in '$(cat ${SSID_FILE})'
+ cico -q node done a899eaf3

POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending email to: obnox@samba.org gd@samba.org anoopcs@cryptolab.net
Finished: FAILURE

Also noticed weird failure notifications sent to registered email addresses.

Subject:
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!

Body:
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS:
Check console output at $BUILD_URL to view the results.

Metadata Update from @arrfab:
- Issue assigned to siddharthvipul1
- Issue marked as depending on: #348
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: centos-ci-infra, high-gain, low-trouble

2 years ago

can you verify now please ? Jenkins was updated, but not the agents, which is normally now done.
So it fixed the issue for #348 so it should also work for you

Yup..it works. I re-triggered a job and is running as expected. I will wait for the email notification to arrive for final confirmation.

Also noticed weird failure notifications sent to registered email addresses.

Subject:
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!

Body:
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS:
Check console output at $BUILD_URL to view the results.

Despite running the job as expected, the email notification still looks weird as above.

Metadata Update from @arrfab:
- Issue assigned to arrfab (was: siddharthvipul1)

2 years ago

unsure about the build notification so something to investigate later.
BTW, migrating to the new openshift setup is recommended to have all projects just having their own dedicated jenkins pod .

Wrt failure notifications, are these just sent without build info ? (like macros/variables not set ?)

Can you verify that it's now working fine ?

Can you verify that it's now working fine ?

Yup. It works now. Thanks

Metadata Update from @anoopcs:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata
Boards 1
CentOS CI Infra Status: Backlog