Here are the AutoSys topics you’ll learn about in this post.
- What is AutoSys?
- AutoSys Features
- CA AutoSys Benefits
- AutoSys is the leader among Job Scheduling Management System Tools?
- Workflow of AutoSys
- Creating AutoSys Jobs
- AutoSys Job Types
- AutoSys Job Status/States – Infographic
- AutoSys ON HOLD vs ON ICE
- Managing File Transfer(FTP/SFTP/SCP) Jobs in AutoSys
- What are AutoSys JIL file/scripts?
- AutoSys Commands – Cheat Sheet
- AutoSys Alarms
- Best Practices For AutoSys Errors/Exception Handling
- AutoSys Exit Codes/Exit Status
What is AutoSys?
AutoSys is a multi-platform automated job control system that provides scheduling, monitoring, and reporting of jobs that reside on any machine connected to a network.
A short video from ca technologies explains, how application dev teams efficiently monitor and manage enterprise workloads, from anywhere at any time, with a customizable dashboard using CA Workload Automation DE.
AutoSys Features :
- This job scheduling software is delivered by CA technologies (Computer Associates).
- CA Workload automation provides end-to-end solution for job management and scheduling.
- Supports database like Microsoft SQL database, Oracle database, Sybase, and IBM DB2 (using DB jobs you can execute SQL commands, SQL queries or any other search query against a database).
- CA AutoSys tool supports scheduling and execution of ETL tool jobs like datastage.
- It is supported on Solaris, AIX, Windows, Linux, HP-UX servers, IBM Mainframes.
- An web server provides run time environment for running web services.
- Supports SSIS/DTS package execution (using dtexec command line utility)
- Other alternate or extra technology’s of AutoSys job scheduling tool’s in market are BMC Control-M, Skybot, Tivoli Workload Scheduler, Flux, Batchman and so on.
- SDLC scrum friendly and offers unicenter service desk
- Latest version of AutoSys is r11.3.6 (also supports java 1.7).
- compatible with AWK programming language.
- Powerful Analytics, Visualization, Simulation and Authorization Capabilities.
CA AutoSys Benefits :
- Increase organization productivity.
- Improve operational efficiency.
- Better reliability and flexibility with less effort and cost.
- Business continuity and Reduced risk of downtime.
AutoSys is the leader among Job Scheduling Management System Tools?
Undoubtedly Yes ! As per the survey conducted by TechValidate “68% of surveyed customers prefer CA Workload Automation over BMC Control-M for visualizing workloads across the enterprise.”
Workflow of AutoSys :
Remote Agent, Event Server, Event Processor are the build blockings of the AutoSys.
Remote Agent : Remote Agent is a process triggered by the event processor to execute a task on remote machine(client machine).
Event Server(AutoSys Database) : Event Server is the repository where respective job events and job definitions are stored.
Event Processor : Event Processor is a crucial component, It reads and processes all the events from the database (event server) and this process can be a window service or unix . Event Processor will scan the database events continously.
Creating AutoSys Jobs:
There are two ways to create an job definitions.
Using AutoSys GUI(AutoSys Graphical User Interface) :Through AutoSys GUI, attributes can be set to describe where, when, and how a job should be run.
Using Job Information Language (JIL) via command-line interface : JIL is a specification language whichs has its own commands to describe where, when, and how a job should be run.
AutoSys Job Types:
Command job : The name itself suggests, i.e Command jobs execute commands. An AutoSys command job is a single command or executable, NT batch file, or unix script/shell script (scripting language).
File Watcher job : The file watcher in AutoSys will watch for the arrival of particular file.
Box (box jobs) : Box jobs are containers that consists other jobs (including other boxes), The box job can be used to control and organize process flow.An impressive feature of this type of job is, boxes can be put another boxes.
AutoSys Job Status/States – Infographic
The tool keeps track of the status of every job. The job status is used to determine when to start other jobs that are dependent on the job.
The following are the list of job status and their respective job change status behaviours.
STARTING : An event is initiated to the start job procedure with the remote agent.
RUNNING : Means the job is in running mode. Incase if the job is a box job, this value simply means that the jobs within the box may be started.
INACTIVE : Means the job is inactive. The job has been never executed or its status was intentionally marked to turnoff its previous completion status.
ACTIVATED : The top-level box that this job is resided is now in RUNNING state, but note that job itself has not started yet.
SUCCESS : The job completed with successfully, by default the exit code ‘0’ is considered as success. In case of box job, success status means that all the jobs inside the box has completed successfully.
FAILURE : The job is failed, which means the exit code is greater than zero.
TERMINATED : The job is terminated while in the running state. A job can be terminated when user sends a KILL job event.
RESTART : The job was unable to start due to application or hardware problems and has been scheduled to restart.
QUE_WAIT : The job is read to run, but there are not enough machine resources available.
ON HOLD : The job is ON HOLD and will not run until it receives job OFF HOLD event.
ON ICE : The job will be ON ICE ( AutoSys jobs on ice) until it receives OFF ICE event.
AutoSys ON HOLD vs ON ICE
Its one of the standard AutoSys interview questions and here are the details which covers ON ICE to OFF ICE vs ON HOLD to OFF HOLD.
- When a job is ON HOLD and If the same job has been kept OFF HOLD, Its runs incase of job starting conditions were met/satisfied.
- When a job is OFF ICE and If the same Job has been kept OFF ICE, Its will not run even though the respective job starting conditions were met/satisfied.
- If a job is ON HOLD, The dependent jobs will not run.
- Where as If a job is ON ICE, the depedent will run as per the schedule.
Managing File Transfer(FTP/SFTP/SCP) Jobs in AutoSys
File Transfer jobs allows transfering of binary, ASCII, and EBCDIC files between an agent computer or remote location and a remote FTP server.
Following file transfer jobs can be defined:
Lets you transfer files using File Transfer Protocol(FTP).
We can securely transfer binary or ASCII files using the Secure File Transfer Protocol (SFTP). Also SFTP protocol supports wildcard transfers, which means you can upload multiple files to a remote FTP server or download multiple files to the agent machine.
We can securely transfer binary files using the Secure Copy Protocol (SCP). Note that SCP protocol doesn’t support wildcard transfers.
What are AutoSys JIL file/scripts?
JIL (Job Information Language) file/scripts are used to built an job definition via command-line interface.
AutoSys JIL scripts contain one or multiple subcommands and one or multiple attribute statements.
The following are the syntax for subcommand and attribute.
sub_command -> Defines a JIL sub command.
object_name -> Defines the name of the object (i,e job / machine) to act on.
attribute_keyword -> Defines a valid JIL attribute.
value -> Defines the setting to apply a attribute.
1. Sample AutoSys JIL file for an command job “helloJob.jil”
machine :unix machine name
command :echo “Hello this a welcome command job”
note: insert job denotes the job name
2. To add the command job “helloJob.jil” in database(db)
Run the following command from unix
jil < echoJob.jil
If you want to read JIL tag and the corresponding job associated with it and load the information via datastage, then you can grep the JIL script.
grep “hello_job:echo” *.jil | cut -d ‘ ‘ -f2
by using above command, you can get the job names, either through Filter options available in Sequential file stage or a script and load using datastage job.
3.To update a job :
Save into a jil file and run it using jil script command jil < filename.jil
insert_job: HelloWorld job_type: c
owner: [email protected]
Let’s go through each attribute from above example
insert_job : Its a JIL keyword which is used as Name to identify a job.
job_type : Its basically defines type of a job aand in this case C refers to a Command Job.
command : This attibute can be any command, UNIX shell script, executable, or batch file.
machine : The client machine on which the command should run.
owner : It will be the userID on which the command will be running.
permission : First letter (g,w,m) combination refers to group user and last letter (x,e) refers to the rights for edit (e) and execute(x).
date_conditions : It enables the run calendar
days_of_week : Days on which the job should run.
start_times : Start time of job that it should run.
alarm_if_fail : If the value is 1, then you will rceive an failure alert and If the value is 0, then you will not recieve any alert.
AutoSys Commands – Cheat Sheet
Following are the list of Autosys commands in unix or any other operating system.
|To convert from CRON to JIL (only in UNIX)||cron2jil|
|To define objects like jobs/machines||jil|
|For event commands||sendevent|
|Check system status||autoflags|
|To define calendars||autocal_asc|
|To maintain Databases||archive_jobs|
|To report job status||autorep|
|To monitor jobs||autosyslog|
|To report job dependencies and conditions||job_depends|
|To manage security||autosys_secure|
|To manage time zone||autotimezone|
Out of above, let me go through some important AutoSys commands that are frequently used.
autocal_asc command :
autocal_asc is used to define calendars.
AutoSys calendars are very helpful incase of job/box specific events, that has to be processed based on a date in a calendar (eg: holiday batch processing based on the date etc).
AutoSys attribute date_conditions is closed related to calendar concepts.
For a job to run ‘standalone’, the ‘Date/Time conditions’ attribute in the ‘Schedule’ section must reflect ‘true’. If the date_conditions field is not set to ‘1’ (true), AutoSys will review ignore any ‘Days’ or ‘Time’ attributes and rely solely of dependencies for scheduling.
If the ‘Date/Time conditions are ‘true’, the job or job box will also reference ‘Run days’, or ‘Run calendar’ and/or ‘Exclude Calendar’; and, ‘Times of day’ or ‘Minutes past hour’. These attributes will be covered in more details in other sections of autosys documentation.
CRON2JIL command :
Cron is a job scheduler in unix based operating systems. It is used to schedule a job (either command or the scripts )that runs at fixed intervals, schedule.
Cron is controlled by a crontab file, a config file that defines shell commands to run cyclically on a specified schedule.
cron2jil command is used to convert from CRON to JIL.
sendevent command :
The sendevent command is a component that sends events to start or stop jobs, stop the AutoSys scheduler, put a job on hold, set a global variable,cancel a scheduled jobs event and so on.
The following are some of the sample sendevent commands, generally it will be denoted by using sendevent e job name.
To start a job : sendevent -E STARTJOB -J
To force start a job : sendevent -E FORCE_STARTJOB -J
To put job ON ICE : sendevent -E ON_ICE -J
To put job OFF ICE : sendevent -E OFF_ICE -J
To kill a job in AutoSys: sendevent -E KILLJOB -J
Event logs can be verified to check the output, errors.
Calling an AutoSys job from Unix Script :
we can call an AutoSys job via unix script, following is an example
JobName : SchedulerCommand
Custom Script Name: callJob.ksh
Server : AutoSys Server
In shell script “callJob.ksh”, use the below sendevent command to pass the event type and job name.
sendevent -E FORCE_STARTJOB -J SchedulerCommand
Now save the unix script and invoke it using command ./callJob.ksh from AutoSys server. thats it, the job ‘SchedulerCommand’ will be successfully force started.
The autorep command is a component that generates reports about jobs, machines, and global variables defined in database. The autorep command pulls data from the database to formulate the reports.
autorep command can be used to achieve the following.
- Displaying the current machine load information.
- Backing up job definitions by extracting definitions and saving them to an output file.
- Displaying a summary of all defined jobs.
- Listing relvant information of any job last run (previous run).
If you want to check the job definition history changes in AutoSys, then you can acieve it using autorep command below
autorep -j “mention the job name”
Also by using wildcards(%) with autorep command, you can export the AutoSys JIL’s.
autorep -j % -q > /temp/filename.jil
autorep Command Syntax:
|command purpose||autorep command syntax|
|To report on jobs in a group||autorep -B group [-d | -s | -q | -o overnum]|
[-I application] [-n] [-w] [-t]
[-R run_num] [-L print_level]
|To report on jobs in an application||autorep -I application [-d | -s | -q | -o overnum]|
[-B group] [-n] [-w] [-t]
[-R run_num] [-L print_level]
|To report on a machine||autorep -M machine [-p | -q] [-d | -s] [-n] [-w]|
|To report on a user-defined job type||autorep -Y job_type [-d | -s | -q ] [-n] [-w]|
|To report on global variables||autorep -G global_name [-n] [-w] [-d | -s]|
|To report on a external instance||autorep -X ext_instance [-s | -q ] [-n] [-w] [-t]|
|To report on a virtual resource||autorep -V virtual_resource_name [-d | -s | -q ] [-n] [-w]|
|To display the version number for the command||autorep -x|
|To display help for the command||autorep -?|
For an instance, AutoSys jobs could be dependent on a arrival file and if the file not yet arrived in an expected time frame, an alarm notification will alert the respective groups or stake holders to take appropriate action.
- Alarms are special events which provides notification mechanism during alarming situations like AutoSys job scheduler failures, dependencies.
- Upon these kind of notifications, it will allow manual interventions to address the issue.
- Alarms are system specific messages about a detected issue.
- Typically alarms are sent through system as a event.
Best Practices For AutoSys Errors/Exception Handling:
Exception Handling is one of the key attribute in the software life cycle and w.r.t AutoSys, it is best practice to have exception handling on failed operator which have exit path for only “retry” scenario. else, having retry without ever exiting the process results in endless loops.
Error/Err files or log files can be found by using following commands and at their respective locations
Scheduler and Application Server logs:
AutoSys Exit Codes/Exit Status
In general, a zero exit code indicates success; while a nonzero exit code indicates an error. Following are exit or error codes which are frequently seen.
Exit Code 15 : The job has been terminated.
Exit Code 101 : CHANGE_STATUS was performed on the job. for eg: The job was changed to FAILURE or TERMINATED status.
Exit Code 121 : Cannot open std_in_file. File does not exist or it is inaccessible.
Exit Code 122 : Cannot open std_out_file. File does not exist or it is inaccessible.
Exit Code 123 : Cannot open std_err_file. File does not exist or it is inaccessible.
Exit Code 127 : Directory that holds the executable is not in the command search PATH.
Exit Code 256 : Unable to execute the command.
Exit Code 512 : Incorrect command options.
-655 SYSTEM_ERROR : STARTJOB failures because auto_remote will not start.
-656 NO_EXIT_CODE : exit_code field in database is initialized to this.
-657 PROCESS_MIA : Set by a chase-generated FAILURE event, for eg: chase cannot find the process.
This Infographic from CA technologies illustrates quantitative and qualitative results of workload automation.
To conclude, here are some genuine CA Workload Automation Reviews from subject matter experts, architects, analysts posted in itcentralstation.com.
Hope you liked it , please feel free to share above autosys reference manual.