Kafka Protocol & KIP
Kafka Protocol & KIP
Links
ApiVersion
KIP-35 - Retrieving protocol version
KAFKA-3304 KIP-35 - Retrieving protocol version
Fix Version/s: 0.10.2.0
KAFKA-3307 Add ApiVersion request/response and server side handling.
Fix Version/s: 0.10.0.0
ApiVersionRequest => ApiKeys
// empty
ApiVersionResponse => ApiVersions
ErrorCode = INT16
ApiVersions = [ApiVersion]
ApiVersion = ApiKey MinVersion MaxVersion
ApiKey = INT16
MinVersion = INT16
MaxVersion = INT16
Metadata
v1
KIP-4 - Metadata Protocol Changes
Released: 0.10.0.0
MetadataRequest => [topics]
Stays the same as version 0 however behavior changes.
In version 0 there was no way to request no topics, and and empty list signified all topics.
In version 1 a null topics list (size -1 on the wire) will indicate that a user wants ALL topic metadata. Compared to an empty list (size 0) which indicates metadata for NO topics should be returned.
MetadataResponse => [brokers] controllerId [topic_metadata]
brokers => node_id host port rack
node_id => INT32
host => STRING
port => INT32
rack => NULLABLE_STRING
controllerId => INT32
topic_metadata => topic_error_code topic is_internal [partition_metadata]
topic_error_code => INT16
topic => STRING
is_internal => BOOLEAN
partition_metadata => partition_error_code partition_id leader [replicas] [isr]
partition_error_code => INT16
partition_id => INT32
leader => INT32
replicas => INT32
isr => INT32
Adds rack, controller_id, and is_internal to the version 0 response.
The behavior of the replicas and isr arrays will be changed in order to support the admin tools, and better represent the state of the cluster:
In version 0, if a broker is down the replicas and isr array will omit the brokers entry and add a REPLICA_NOT_AVAILABLE error code.
In version 1, no error code will be set and a the broker id will be included in the replicas and isr array.
Note: A user can still detect if the replica is not available, by checking if the broker is in the returned broker list.
v2
Released: 0.10.1.0
Metadata Request (Version: 2) => [topics]
topics => STRING
Metadata Response (Version: 2) => cluster_id [brokers] controller_id [topic_metadata]
cluster_id => STRING
brokers => node_id host port rack
node_id => INT32
host => STRING
port => INT32
rack => NULLABLE_STRING
controller_id => INT32
topic_metadata => topic_error_code topic is_internal [partition_metadata]
topic_error_code => INT16
topic => STRING
is_internal => BOOLEAN
partition_metadata => partition_error_code partition_id leader [replicas] [isr]
partition_error_code => INT16
partition_id => INT32
leader => INT32
replicas => INT32
isr => INT32
CreateTopics/DeleteTopic
KIP-4 - Command line and centralized administrative operations
KAFKA-2945 CreateTopic - protocol and server side implementation
Fix Version/s: 0.10.1.0
CreateTopics Request (Version: 0) => [create_topic_requests] timeout
create_topic_requests => topic num_partitions replication_factor [replica_assignment] [configs]
topic => STRING
num_partitions => INT32
replication_factor => INT16
replica_assignment => partition_id [replicas]
partition_id => INT32
replicas => INT32
configs => config_key config_value
config_key => STRING
config_value => STRING
timeout => INT32
CreateTopics Response (Version: 0) => [topic_error_codes]
topic_error_codes => topic error_code
topic => STRING
error_code => INT16
KAFKA-2946 DeleteTopic - protocol and server side implementation
Fix Version/s: 0.10.1.0
DeleteTopics Request (Version: 0) => [topics] timeout
topics => STRING
timeout => INT32
DeleteTopics Response (Version: 0) => [topic_error_codes]
topic_error_codes => topic error_code
topic => STRING
error_code => INT16
DescribeConfigs/AlterConfigs
KIP-133: Describe and Alter Configs Admin APIs
Released: 0.11.0.0
DescribeConfigs Request (Version: 0) => [resource [config_name]]
resource => resource_type resource_name
resource_type => INT8
resource_name => STRING
config_name => STRING
DescribeConfigs Response (Version: 0) => error_code [entities]
entities => error_code resource_type resource_name [configs]
error_code => INT16
resource_type => INT8
resource_name => STRING
configs =>
config_name => STRING
config_value => STRING
read_only => BOOLEAN
is_default => BOOLEAN
is_sensitive => BOOLEAN
AlterConfigs Request (Version: 0) => [resources] validate_only
validate_only => BOOLEAN
resources => resource_type resource_name [configs]
resource_type => INT8
resource_name => STRING
configs =>
config_name => STRING
config_value => STRING
AlterConfigs Response (Version: 0) => [responses]
responses => resource_type resource_name error_code error_message
resource_type => INT8
resource_name => STRING
error_code => INT16
error_message => STRING
ListGroups/DescribeGroup
KIP-40: ListGroups and DescribeGroup
Released: 0.9.0.0
ListGroupsRequest =>
ListGroupsResponse => [GroupId ProtocolType]
GroupId => string
ProtocolType => string
DescribeGroupRequest => GroupId
GroupId => string
DescribeGroupResponse => ErrorCode State ProtocolType Protocol Leader Members
ErrorCode => string
State => string
ProtocolType => string
Protocol => string
Leader => string
Members => [MemberId Host ClientId MemberMetadata MemberAssignment]
MemberId => string
Host => string
ClientId => string
MemberMetadata => bytes
MemberAssignment => bytes