Enable forward compatibility for AutoML models
Adopt v10.2 AutoML Worker or model to IA Cloud v10.1.6
For OOTB or models based on Python 2, no additional action is required. Everything is supposed to work by default.
For models based on Python 3, proceed as described below:
Install Python 3.7 on your Agents.
Expand for installation code
``` sudo yum -y install yum-utils sudo yum -y groupinstall development sudo yum -y install \ wget \ zlib-devel \ gcc \ zlib-devel \ bzip2 \ bzip2-devel \ readline-devel \ sqlite \ sqlite-devel \ openssl-devel \ tk-devel \ libffi-devel \ xz-devel parent_directory=/usr/local/python mkdir -p $parent_directory cd $parent_directory wget https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tgz tar -zxvf Python-3.7.9.tgz cd Python-3.7.9 ./configure --prefix=$parent_directory/python3.7.9 --enable-optimizations make clean make make install cd $parent_directory/python3.7.9/bin ln -s python3.7 python ln -s pip3.7 pip new_shebang='#!/usr/bin/env python3.7' escaped_new_shebang=$(printf '%s\n' "$new_shebang" | sed -e 's/[\/&]/\\&/g') sed -i "1s/.*/$escaped_new_shebang/" pip cd $parent_directory zip -r python3.7.9.zip python3.7.9/ ```Add Python dependencies:
python3.7 -m pip install six==1.15.0 python3.7 -m pip install numpy==1.18.5 python3.7 -m pip install spacy==2.3.2 python3.7 -m pip install scikit-learn==0.23.1 python3.7 -m pip install mxnet==1.6.0
Add the following variables to
/opt/workfusion/environment.sh:export PYTHONHOME= export PYTHONPATH= export PATH=
The final environment.sh file should look like this:
environment.sh [-M--] 32 L:[ 1+33 34/ 34] *(2175/2239b) 0109 0x06D
export PYTHONHOME=/usr/local/python/python3.7.9/
export PATH=/usr/local/python/python3.7.9/bin:$PATH
export PYTHONPATH=/usr/local/python/python3.7.9/lib:/usr/local/python/python3.7.9/lib64
export PATH=/opt/workfusion/rpms/usr/bin:/opt/workfusion/rpms/usr/sbin:$PATH
export LD_LIBRARY_PATH=/opt/workfusion/rpms/usr/lib:/opt/workfusion/rpms/usr/lib64:/opt/workfusion/rpms/lib64/openssl/engines:/opt/workfusion/rpms/libexec/coreutils:$LD_LIBRARY_PATH
export PIP_NO_CACHE_DIR=0
export JAVA_HOME=/opt/workfusion/java
export PATH=/opt/workfusion/java/bin:$PATH
export FONTCONFIG_FILE=/opt/workfusion/rpms/etc/fonts/fonts.conf
export PATH=/opt/workfusion/liquibase:$PATH
export PATH=/opt/workfusion/vowpal-wabbit/bin:$PATH
export LD_LIBRARY_PATH=/opt/workfusion/vowpal-wabbit/lib:$LD_LIBRARY_PATH
export PATH=/opt/workfusion/esuit/gs:$PATH
export PATH=/opt/workfusion/esuit/im:$PATH
export PATH=/opt/workfusion/esuit/libtiff_tools/usr/bin:$PATH
export MANPATH=$(manpath -q):/opt/workfusion/esuit/libtiff_tools/usr/share/man/man1/
LD_LIBRARY_PATH=/opt/workfusion/mesos/usr/lib/:/opt/workfusion/mesos/usr/lib64/:$LD_LIBRARY_PATH
Adopt v10.1.6 AutoML Worker or model to IA Cloud v10.1.4
At the moment, forward compatibility is not supported for 10.1.4 version. You get the following exception:
Expand to see the log file
2021-01-27 14:46:12.292 INFO s.c.a.AnnotationConfigApplicationContext Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@79b4d0f: startup date [Wed Jan 27 14:46:12 UTC 2021]; root of context hierarchy
2021-01-27 14:46:13.042 INFO f.a.AutowiredAnnotationBeanPostProcessor JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2021-01-27 14:46:13.177 INFO trationDelegate$BeanPostProcessorChecker Bean 'configurationPropertiesRebinderAutoConfiguration' of type [org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$755a46de] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
_____ _ _____ __
| _ | _ _ | |_ ___ | || | Auto ML Core:
| || | || _|| . || | | || |__ Spring Boot: 2.0.4.RELEASE
|__|__||___||_| |___||_|_|_||_____|
2021-01-27 14:46:14.847 INFO b.c.PropertySourceBootstrapConfiguration Located property source: CompositePropertySource {name='zookeeper', propertySources=[ZookeeperPropertySource {name='config/com.wf.test:ie-model-10-1-6-1:2.0-develop:execution,logstash'}, ZookeeperPropertySource {name='config/com.wf.test:ie-model-10-1-6-1:2.0-develop:execution,execute'}, ZookeeperPropertySource {name='config/com.wf.test:ie-model-10-1-6-1:2.0-develop:execution,execution-worker'}, ZookeeperPropertySource {name='config/com.wf.test:ie-model-10-1-6-1:2.0-develop:execution'}, ZookeeperPropertySource {name='config/application,logstash'}, ZookeeperPropertySource {name='config/application,execute'}, ZookeeperPropertySource {name='config/application,execution-worker'}, ZookeeperPropertySource {name='config/application'}]}
2021-01-27 14:46:14.989 INFO com.workfusion.vds.VdsBootApplication The following profiles are active: execution-worker,execute,logstash
2021-01-27 14:46:14.993 INFO ConfigServletWebServerApplicationContext Refreshing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@20d3d15a: startup date [Wed Jan 27 14:46:14 UTC 2021]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@79b4d0f
2021-01-27 14:46:17.126 INFO o.s.c.a.ConfigurationClassParser Properties location [classpath:application.properties] not resolvable: class path resource [application.properties] cannot be opened because it does not exist
2021-01-27 14:46:17.832 INFO c.e.c.c.NoRegisteredCuratorBeanCondition Curator framework bean is already registered : false
2021-01-27 14:46:19.061 INFO o.s.cloud.context.scope.GenericScope BeanFactory id=6fd29f68-33a7-3e69-b50a-0c8d7c7b0cb1
2021-01-27 14:46:19.215 INFO f.a.AutowiredAnnotationBeanPostProcessor JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2021-01-27 14:46:19.420 INFO trationDelegate$BeanPostProcessorChecker Bean 'org.springframework.amqp.rabbit.annotation.RabbitBootstrapConfiguration' of type [org.springframework.amqp.rabbit.annotation.RabbitBootstrapConfiguration$$EnhancerBySpringCGLIB$$1fac020f] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-01-27 14:46:19.506 INFO trationDelegate$BeanPostProcessorChecker Bean 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration' of type [org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$755a46de] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-01-27 14:46:20.744 INFO o.s.b.w.embedded.tomcat.TomcatWebServer Tomcat initialized with port(s): 41071 (http)
2021-01-27 14:46:20.826 INFO org.apache.catalina.core.StandardEngine Starting Servlet Engine: Apache Tomcat/8.5.32
2021-01-27 14:46:20.974 INFO o.a.catalina.core.AprLifecycleListener The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/opt/workfusion/mesos/usr/lib/:/opt/workfusion/mesos/usr/lib64/:/opt/workfusion/vowpal-wabbit/lib:/opt/workfusion/rpms/usr/lib:/opt/workfusion/rpms/usr/lib64:/opt/workfusion/rpms/lib64/openssl/engines:/opt/workfusion/rpms/libexec/coreutils:/opt/workfusion/mesos/usr/lib:/opt/workfusion/mesos/usr/lib64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
2021-01-27 14:46:21.285 INFO o.a.c.c.C.[Tomcat].[localhost].[/] Initializing Spring embedded WebApplicationContext
2021-01-27 14:46:21.296 INFO o.s.web.context.ContextLoader Root WebApplicationContext: initialization completed in 6303 ms
2021-01-27 14:46:23.936 INFO o.s.b.w.servlet.FilterRegistrationBean Mapping filter: 'characterEncodingFilter' to: [/*]
2021-01-27 14:46:23.958 INFO o.s.b.w.servlet.FilterRegistrationBean Mapping filter: 'webMvcMetricsFilter' to: [/*]
2021-01-27 14:46:23.959 INFO o.s.b.w.servlet.FilterRegistrationBean Mapping filter: 'hiddenHttpMethodFilter' to: [/*]
2021-01-27 14:46:23.970 INFO o.s.b.w.servlet.FilterRegistrationBean Mapping filter: 'httpPutFormContentFilter' to: [/*]
2021-01-27 14:46:23.970 INFO o.s.b.w.servlet.FilterRegistrationBean Mapping filter: 'requestContextFilter' to: [/*]
2021-01-27 14:46:23.971 INFO o.s.b.w.servlet.FilterRegistrationBean Mapping filter: 'httpTraceFilter' to: [/*]
2021-01-27 14:46:23.971 INFO o.s.b.w.servlet.ServletRegistrationBean Servlet dispatcherServlet mapped to [/]
LOGBACK: No context given for com.workfusion.spa.core.execution.metrics.logstash.MetricsLogstashSocketAppender[null]
2021-01-27 14:46:24.568 INFO xecutionWorkerWithDeathPillConfiguration Init death pill counter '/kill/death-pill/automl-model-service/com.wf.test/ie-model-10-1-6-1/2.0-develop/execution/ie-model-10-1-6-1-2/release-10-1-4-1-bepagent1.workfusion.com/death-pill-count'
2021-01-27 14:46:25.470 INFO o.s.w.s.handler.SimpleUrlHandlerMapping Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2021-01-27 14:46:26.328 INFO s.w.s.m.m.a.RequestMappingHandlerAdapter Looking for @ControllerAdvice: org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@20d3d15a: startup date [Wed Jan 27 14:46:14 UTC 2021]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@79b4d0f
2021-01-27 14:46:26.541 INFO s.w.s.m.m.a.RequestMappingHandlerMapping Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.servlet.error.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2021-01-27 14:46:26.561 INFO s.w.s.m.m.a.RequestMappingHandlerMapping Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.servlet.error.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
2021-01-27 14:46:26.630 INFO o.s.w.s.handler.SimpleUrlHandlerMapping Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2021-01-27 14:46:26.631 INFO o.s.w.s.handler.SimpleUrlHandlerMapping Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2021-01-27 14:46:26.755 INFO o.s.b.a.w.s.WelcomePageHandlerMapping Adding welcome page: class path resource [static/index.html]
2021-01-27 14:46:26.806 INFO c.w.r.l.LogLevelUpdateAutoConfiguration Zookeeper node to watch: /config/com.wf.test:ie-model-10-1-6-1:2.0-develop:execution/log
2021-01-27 14:46:26.969 ERROR o.a.c.f.recipes.cache.PathChildrenCache
org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /config/com.wf.test:ie-model-10-1-6-1:2.0-develop:execution
at org.apache.zookeeper.KeeperException.create(KeeperException.java:106)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1733)
at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:351)
at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:230)
at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:224)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:93)
at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:221)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:206)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:35)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.createContainers(CuratorFrameworkImpl.java:261)
at org.apache.curator.framework.EnsureContainers.internalEnsure(EnsureContainers.java:69)
at org.apache.curator.framework.EnsureContainers.ensure(EnsureContainers.java:53)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.ensurePath(PathChildrenCache.java:589)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.refresh(PathChildrenCache.java:494)
at org.apache.curator.framework.recipes.cache.RefreshOperation.invoke(RefreshOperation.java:35)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$8.run(PathChildrenCache.java:801)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-01-27 14:46:28.449 INFO o.s.b.a.e.web.EndpointLinksResolver Exposing 6 endpoint(s) beneath base path '/actuator'
2021-01-27 14:46:28.486 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/health],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.496 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/env/{toMatch}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.496 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/env],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.506 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/env],methods=[POST],consumes=[application/vnd.spring-boot.actuator.v2+json || application/json],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.506 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/env],methods=[DELETE],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.516 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/info],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.516 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/loggers/{name}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.517 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/loggers/{name}],methods=[POST],consumes=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.517 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/loggers],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.528 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/threaddump],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.528 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/metrics],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.529 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator/metrics/{requiredMetricName}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2021-01-27 14:46:28.530 INFO s.b.a.e.w.s.WebMvcEndpointHandlerMapping Mapped "{[/actuator],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto protected java.util.Map<java.lang.String, java.util.Map<java.lang.String, org.springframework.boot.actuate.endpoint.web.Link>> org.springframework.boot.actuate.endpoint.web.servlet.WebMvcEndpointHandlerMapping.links(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2021-01-27 14:46:28.642 INFO o.s.j.e.a.AnnotationMBeanExporter Registering beans for JMX exposure on startup
2021-01-27 14:46:28.663 INFO o.s.j.e.a.AnnotationMBeanExporter Bean with name 'getConnectionFactory' has been autodetected for JMX exposure
2021-01-27 14:46:28.678 INFO o.s.j.e.a.AnnotationMBeanExporter Bean with name 'environmentManager' has been autodetected for JMX exposure
2021-01-27 14:46:28.679 INFO o.s.j.e.a.AnnotationMBeanExporter Bean with name 'configurationPropertiesRebinder' has been autodetected for JMX exposure
2021-01-27 14:46:28.691 INFO o.s.j.e.a.AnnotationMBeanExporter Bean with name 'refreshScope' has been autodetected for JMX exposure
2021-01-27 14:46:28.694 INFO o.s.j.e.a.AnnotationMBeanExporter Located managed bean 'environmentManager': registering with JMX server as MBean [org.springframework.cloud.context.environment:name=environmentManager,type=EnvironmentManager]
2021-01-27 14:46:28.713 INFO o.s.j.e.a.AnnotationMBeanExporter Located managed bean 'refreshScope': registering with JMX server as MBean [org.springframework.cloud.context.scope.refresh:name=refreshScope,type=RefreshScope]
2021-01-27 14:46:28.727 INFO o.s.j.e.a.AnnotationMBeanExporter Located managed bean 'getConnectionFactory': registering with JMX server as MBean [org.springframework.amqp.rabbit.connection:name=getConnectionFactory,type=CachingConnectionFactory]
2021-01-27 14:46:28.770 INFO o.s.j.e.a.AnnotationMBeanExporter Located managed bean 'configurationPropertiesRebinder': registering with JMX server as MBean [org.springframework.cloud.context.properties:name=configurationPropertiesRebinder,context=20d3d15a,type=ConfigurationPropertiesRebinder]
2021-01-27 14:46:28.993 ERROR o.a.c.f.recipes.cache.PathChildrenCache
org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /config/com.wf.test:ie-model-10-1-6-1:2.0-develop:execution
at org.apache.zookeeper.KeeperException.create(KeeperException.java:106)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1733)
at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:351)
at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:230)
at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:224)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:93)
at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:221)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:206)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:35)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.createContainers(CuratorFrameworkImpl.java:261)
at org.apache.curator.framework.EnsureContainers.internalEnsure(EnsureContainers.java:69)
at org.apache.curator.framework.EnsureContainers.ensure(EnsureContainers.java:53)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.ensurePath(PathChildrenCache.java:589)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.refresh(PathChildrenCache.java:494)
at org.apache.curator.framework.recipes.cache.RefreshOperation.invoke(RefreshOperation.java:35)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$8.run(PathChildrenCache.java:801)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-01-27 14:46:29.108 INFO o.s.c.support.DefaultLifecycleProcessor Starting beans in phase 2147483647
2021-01-27 14:46:29.126 INFO s.a.ScheduledAnnotationBeanPostProcessor No TaskScheduler/ScheduledExecutorService bean found for scheduled processing
2021-01-27 14:46:29.341 INFO o.s.b.w.embedded.tomcat.TomcatWebServer Tomcat started on port(s): 41071 (http) with context path ''
2021-01-27 14:46:29.344 INFO com.workfusion.vds.VdsBootApplication Started VdsBootApplication in 20.523 seconds (JVM running for 21.691)
2021-01-27 14:46:29.347 INFO c.w.vds.event.ApplicationReadyListener ApplicationReadyEvent. BEP worker is starting to launch required component.
2021-01-27 14:46:29.347 INFO c.w.s.c.e.w.c.BepWorkerLauncher Start bep components
2021-01-27 14:46:29.348 INFO c.w.s.c.e.w.router.TaskInputService Waiting for the 'exec.automl-model-service.worker.com.wf.test.ie-model-10-1-6-1.2.0-develop.execution.ie-model-10-1-6-1-2' queue to appear
2021-01-27 14:46:29.367 INFO o.s.a.r.c.CachingConnectionFactory Attempting to connect to: [release-10-1-4-1-ct1.workfusion.com:5672]
2021-01-27 14:46:29.518 INFO o.s.a.r.c.CachingConnectionFactory Created new connection: getConnectionFactory#4af0ac59:0/SimpleConnection@1536602f [delegate=amqp://bep@10.214.75.75:5672/bep, localPort= 44208]
2021-01-27 14:46:29.599 INFO c.w.s.c.e.w.router.TaskInputService Starting listener for the 'exec.automl-model-service.worker.com.wf.test.ie-model-10-1-6-1.2.0-develop.execution.ie-model-10-1-6-1-2' queue
2021-01-27 14:46:29.724 INFO c.w.s.c.e.w.c.BepWorkerLauncher Start death pill counter
2021-01-27 14:46:29.771 WARN c.w.s.c.e.amqp.logging.LoggingUtils Message from queue exec.automl-model-service.worker.com.wf.test.ie-model-10-1-6-1.2.0-develop.execution.ie-model-10-1-6-1-2 doesn't have required header _taskCorrelationId
2021-01-27 14:46:29.839 INFO c.w.s.c.e.worker.router.TaskProcessor Incoming Task - Type: execution-task, Id: 92f64317-c653-464c-92fb-2e7e50d48f43
2021-01-27 14:46:29.852 INFO w.a.w.e.PreparationModelExecutionService Start copying model files for execution. Source: /opt/workfusion/vds-data/execution/ie-model-10-1-6-1-2/, Destination: /opt/workfusion/vds-data/workers/process/worker/automl-model-service/com-wf-test/ie-model-10-1-6-1/2-0-develop/execution/ie-model-10-1-6-1-2/worker-0127-144607-193/
2021-01-27 14:46:30.116 INFO w.a.w.e.PreparationModelExecutionService Copying model files for execution completed
2021-01-27 14:46:30.117 INFO c.w.s.c.e.worker.router.TaskProcessor Routing task to com.workfusion.nlp.uima.workflow.async.worker.execution.ModelExecutionWorker executor
2021-01-27 14:46:30.388 ERROR o.s.boot.SpringApplication Application run failed
org.springframework.beans.factory.BeanInitializationException: Death pill shared count can't be initialized; nested exception is org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /kill/death-pill/automl-model-service/com.wf.test/ie-model-10-1-6-1/2.0-develop/execution/ie-model-10-1-6-1-2/release-10-1-4-1-bepagent1.workfusion.com/death-pill-count
at com.workfusion.spa.core.execution.worker.configuration.BepWorkerLauncher.launchBepComponents(BepWorkerLauncher.java:47)
at com.workfusion.vds.event.ApplicationReadyListener.onApplicationEvent(ApplicationReadyListener.java:26)
at com.workfusion.vds.event.ApplicationReadyListener.onApplicationEvent(ApplicationReadyListener.java:13)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:400)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:354)
at org.springframework.boot.context.event.EventPublishingRunListener.running(EventPublishingRunListener.java:103)
at org.springframework.boot.SpringApplicationRunListeners.running(SpringApplicationRunListeners.java:78)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:346)
at com.workfusion.vds.VdsBootApplication.main(VdsBootApplication.java:26)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /kill/death-pill/automl-model-service/com.wf.test/ie-model-10-1-6-1/2.0-develop/execution/ie-model-10-1-6-1-2/release-10-1-4-1-bepagent1.workfusion.com/death-pill-count
at org.apache.zookeeper.KeeperException.create(KeeperException.java:106)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1836)
at org.apache.curator.framework.imps.CreateBuilderImpl$16.call(CreateBuilderImpl.java:1131)
at org.apache.curator.framework.imps.CreateBuilderImpl$16.call(CreateBuilderImpl.java:1113)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:93)
at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:1110)
at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:593)
at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:583)
at org.apache.curator.framework.imps.CreateBuilderImpl$4.forPath(CreateBuilderImpl.java:443)
at org.apache.curator.framework.imps.CreateBuilderImpl$4.forPath(CreateBuilderImpl.java:379)
at org.apache.curator.framework.recipes.shared.SharedValue.start(SharedValue.java:256)
at org.apache.curator.framework.recipes.shared.SharedCount.start(SharedCount.java:163)
at com.workfusion.spa.core.execution.worker.configuration.BepWorkerLauncher.launchBepComponents(BepWorkerLauncher.java:45)
... 19 common frames omitted
2021-01-27 14:46:30.390 INFO ConfigServletWebServerApplicationContext Closing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@20d3d15a: startup date [Wed Jan 27 14:46:14 UTC 2021]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@79b4d0f
2021-01-27 14:46:30.423 INFO o.s.c.support.DefaultLifecycleProcessor Stopping beans in phase 2147483647
2021-01-27 14:46:30.423 INFO o.s.c.support.DefaultLifecycleProcessor Stopping beans in phase 0
2021-01-27 14:46:30.533 INFO o.s.j.e.a.AnnotationMBeanExporter Unregistering JMX-exposed beans on shutdown
2021-01-27 14:46:30.533 INFO o.s.j.e.a.AnnotationMBeanExporter Unregistering JMX-exposed beans
2021-01-27 14:46:30.536 INFO o.s.a.r.l.SimpleMessageListenerContainer Waiting for workers to finish.
2021-01-27 14:46:31.775 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:31.795 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:31.796 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:31.809 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:31.810 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:31.810 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:31.810 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:31.814 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:31.814 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:31.814 WARN c.w.nlp.CSVReaderAccountExtractorRule Account number group='' isn't numeric
2021-01-27 14:46:35.533 INFO o.s.a.r.l.SimpleMessageListenerContainer Successfully waited for workers to finish.
2021-01-27 14:46:35.534 INFO c.w.s.c.e.w.router.TaskInputService Router was stopped.
2021-01-27 14:46:36.732 WARN o.a.c.loader.WebappClassLoaderBase The web application [ROOT] appears to have started a thread named [Death Pill Listener] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)