If you want to initialize multiple variables, then they need to be of the same type, and you have to use just one declaration, so it would look like this:
for (int i = 1, k = 2; i<=4000000; i++ ) {
}
I wrote a bit about the syntax of for loops in response to Types permitted in for loop variable declarations? This isn't quite the same question, but the answer makes sense here, too I think.
In general, you can have a look at the Java language specification for the for statement. You can declare and initialize any type of variable in a for loop, and can even declare multiple variables, so long as they're all the same type. The relevant productions in the grammar are:
BasicForStatement:
for ( ForInitopt ; Expressionopt ; ForUpdateopt ) Statement
ForInit:
StatementExpressionList
LocalVariableDeclaration
LocalVariableDeclaration:
VariableModifiersopt Type VariableDeclarators
VariableDeclarators:
VariableDeclarator
VariableDeclarators , VariableDeclarator
This means that you can do any of the following, e.g.,
for ( ; … ; … ) // no variable declaration at all
for ( int i; … ; … ) // variable declaration with no initial value
for ( int i=0; … ; … ) // variable declaration with initial value
for ( int i=0, j=1; … ; … ) // multiple variables
for ( final Iterator<T> it = …; … ; … ) // final variable
The fourth case is the one that you're concerned with at the moment.
As an aside, the ForInit can also be a StatementExpressionList, which means that instead of declaring and initializing variables, you can also just execute some statements. E.g, you could do this (but this isn't a particularly useful example):
for ( System.out.println( "beginning loop" ; … ; … )