Kafka Loader Loading Job Management

Kafka Loader loading jobs are managed the same way as regular loading jobs. The three key commands are

  • SHOW LOADING STATUS

  • ABORT LOADING JOB

  • RESUME LOADING JOB

For example, the syntax for the SHOW LOADING STATUS command is as follows:

SHOW LOADING STATUS job_id|ALL

To refer to a specific job instance, using the job_id which is provided when RUN LOADING JOB is executed. For each loading job, the above command reports the following information :

  • Current loaded offset for each partition

  • Average loading speed

  • Loaded size

  • Duration

See Inspecting and Managing Loading Jobs for more details.

Enable concurrent loading jobs

By default, only one loading job created with Kafka Loader is allowed to be running. You can increase this limit to enable concurrently by changing the following configuration:

$ gadmin config set KafkaLoader.ReplicaNumber <new_limit>
$ gadmin config apply -y && gadmin restart gse gpe restpp -y