Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve container credentials retrieval matching container repository names #278

Open
wants to merge 49 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
aa06fdf
added repository parser
munishchouhan Aug 8, 2023
f081df9
added repository login
munishchouhan Aug 8, 2023
138d65f
Merge remote-tracking branch 'origin/master' into 224-improve-contain…
munishchouhan Aug 8, 2023
521cb46
reused some logic
munishchouhan Aug 8, 2023
5c7630c
UTs fixed
munishchouhan Aug 8, 2023
1622402
formatted
munishchouhan Aug 9, 2023
ed5dc83
formatted
munishchouhan Aug 9, 2023
fa131a0
best match algo added
munishchouhan Aug 10, 2023
2ae164f
'container-reg' check added in best match algo
munishchouhan Aug 10, 2023
c559360
Merge remote-tracking branch 'origin/master' into 224-improve-contain…
munishchouhan Aug 11, 2023
50c52f1
removed repository login
munishchouhan Aug 11, 2023
5712f43
Merge remote-tracking branch 'origin/master' into 224-improve-contain…
munishchouhan Aug 16, 2023
69f6925
Merge remote-tracking branch 'origin/master' into 224-improve-contain…
munishchouhan Aug 21, 2023
f9072ab
Merge remote-tracking branch 'origin/master' into 224-improve-contain…
munishchouhan Aug 25, 2023
99b9d2c
Merge remote-tracking branch 'origin/master' into 224-improve-contain…
munishchouhan Aug 28, 2023
821005c
Merge remote-tracking branch 'origin/master' into 224-improve-contain…
munishchouhan Aug 31, 2023
2a1ef2a
merge master
munishchouhan Sep 8, 2023
5b2aaa9
minor change
munishchouhan Sep 8, 2023
045067b
minor change
munishchouhan Sep 8, 2023
bc84cb5
Merge remote-tracking branch 'origin/master' into 224-improve-contain…
munishchouhan Sep 15, 2023
e59fc14
master merged
munishchouhan Oct 10, 2023
b2c4e98
Merge branch 'master' into 224-improve-container-credentials-retrieva…
pditommaso Oct 29, 2023
541ff1a
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Nov 7, 2023
92dfedc
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Nov 13, 2023
7eba402
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Nov 17, 2023
7c55158
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Nov 20, 2023
23c87bf
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Nov 21, 2023
36aac46
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Nov 27, 2023
63ee00e
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Nov 29, 2023
6260b55
Merge branch 'master' into 224-improve-container-credentials-retrieva…
pditommaso Dec 20, 2023
1468dcc
Merge branch 'master' into 224-improve-container-credentials-retrieva…
pditommaso Dec 20, 2023
2dd789b
wip
pditommaso Dec 21, 2023
f8eed91
wip2
pditommaso Dec 21, 2023
57a1d08
wip3
pditommaso Dec 21, 2023
c59f5d8
wip4
pditommaso Dec 21, 2023
4fbf3bb
wip5
pditommaso Dec 21, 2023
bfb4ab4
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Jan 9, 2024
259657f
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Jan 11, 2024
9d78d42
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Jan 16, 2024
3d74734
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Jan 29, 2024
d5d09ad
master merged
munishchouhan Apr 1, 2024
880ed7c
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Apr 4, 2024
6757b10
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan May 30, 2024
7bb7504
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Jun 11, 2024
689a425
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Jul 9, 2024
42e7a0a
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Aug 30, 2024
d0709aa
fixed errors
munishchouhan Aug 30, 2024
f412a8f
fixed tests
munishchouhan Aug 30, 2024
807aecf
Merge branch 'master' into 224-improve-container-credentials-retrieva…
munishchouhan Oct 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 88 additions & 6 deletions src/main/groovy/io/seqera/wave/auth/RegistryAuthServiceImpl.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,11 @@ class RegistryAuthServiceImpl implements RegistryAuthService {
// 0. default to 'docker.io' when the registry name is empty
if( !registryName )
registryName = DOCKER_IO
//check if its a repository or a registry
RepositoryInfo repositoryInfo = parseURI(registryName)
if(repositoryInfo.repository) {
registryName = repositoryInfo.registry
}
munishchouhan marked this conversation as resolved.
Show resolved Hide resolved

// 1. look up the registry authorisation info for the given registry name
final registry = lookupService.lookup(registryName)
Expand All @@ -99,12 +104,18 @@ class RegistryAuthServiceImpl implements RegistryAuthService {
// 3. make a request against the authorization "realm" service using basic
// credentials to get the login token
final basic = "${creds.username}:${creds.password}".bytes.encodeBase64()
final endpoint = registry.auth.endpoint
def endpoint = registry.auth.endpoint

if(repositoryInfo.repository) {
endpoint = new URI("${endpoint}&scope=repository:${repositoryInfo.repository}:pull")
}

HttpRequest request = HttpRequest.newBuilder()
.uri(endpoint)
.GET()
.header("Authorization", "Basic $basic")
.build()

// retry strategy
final retryable = Retryable
.of(httpConfig)
Expand All @@ -114,7 +125,13 @@ class RegistryAuthServiceImpl implements RegistryAuthService {

if( response.statusCode() == 200 ) {
log.debug "Container registry '$endpoint' login - response: ${StringUtils.trunc(response.body())}"
return true

if(repositoryInfo.repository){
// 4. make a request to repository using bearer
return loginToRepository(registry.host, repositoryInfo.repository, parseToken(response.body()))
}else{
return true
}
}
else {
log.warn "Container registry '$endpoint' login FAILED: ${response.statusCode()} - response: ${StringUtils.trunc(response.body())}"
Expand Down Expand Up @@ -194,10 +211,7 @@ class RegistryAuthServiceImpl implements RegistryAuthService {
HttpResponse<String> resp = retryable.apply(()-> httpClient.send(req, HttpResponse.BodyHandlers.ofString()))
final body = resp.body()
if( resp.statusCode()==200 ) {
final result = (Map) new JsonSlurper().parseText(body)
// note: azure registry returns 'access_token'
// see also specs https://docs.docker.com/registry/spec/auth/token/#requesting-a-token
final token = result.get('token') ?: result.get('access_token')
final token = parseToken(body)
if( token ) {
log.trace "Registry auth '$login' => token: ${StringUtils.redact(token)}"
return token
Expand All @@ -207,6 +221,18 @@ class RegistryAuthServiceImpl implements RegistryAuthService {
throw new RegistryUnauthorizedAccessException("Unable to authorize request: $login", resp.statusCode(), body)
}

/**
* Implements a parser to extract token
*
* @param body, body of HTTPResponse
*/
String parseToken(String body){
final result = (Map) new JsonSlurper().parseText(body)
// note: azure registry returns 'access_token'
// see also specs https://docs.docker.com/registry/spec/auth/token/#requesting-a-token
return result.get('token') ?: result.get('access_token')
}

String buildLoginUrl(URI realm, String image, String service){
String result = "${realm}?scope=repository:${image}:pull"
if(service) {
Expand Down Expand Up @@ -239,5 +265,61 @@ class RegistryAuthServiceImpl implements RegistryAuthService {
cacheTokens.invalidate(key)
}

/**
* Implements a parser to get registry and repository name from a URI
*
* @param endpoint, repository URL e.g. https://docker.io/hrma017/dev
*/
protected RepositoryInfo parseURI(String endpoint){
RepositoryInfo repositoryInfo = new RepositoryInfo()
if(endpoint.startsWith("https://")){
endpoint = endpoint.replace("https://","")
}else if(endpoint.startsWith("http://")){
endpoint = endpoint.replace("http://","")
}
def parts = endpoint.split("/")
if(parts.length>1){
repositoryInfo.registry = parts[0]
StringBuilder repo =new StringBuilder(parts[1])
for(int i =2;i<parts.length;i++){
repo.append("/"+parts[i])
}
repositoryInfo.repository = repo
}

return repositoryInfo
}
/**
* Implements container repository login
*
* @param host The repository host e.g. https://registry-1.docker.io
* @param repository is repository name e.g. org/image
* @param token, token in the response of registry login
*/
boolean loginToRepository(URI host, String repository, String token){
URI endpoint = lookupService.registryEndpoint(host.toString())
final repositoryEndpoint = "$endpoint/${repository}/tags/list"

// Use the access token to access the repository
HttpRequest repositoryRequest = HttpRequest.newBuilder()
.uri(URI.create(repositoryEndpoint))
.GET()
.header("Authorization", "Bearer " + token)
.build();

final retryable = Retryable
.of(httpConfig)
.onRetry((event) -> log.warn("Unable to connect '$repositoryEndpoint' - attempt: ${event.attemptCount}; cause: ${event.lastFailure.message}"))
//make a request
HttpResponse<String> repositoryResponse = retryable.apply(()->httpClient.send(repositoryRequest, HttpResponse.BodyHandlers.ofString()));

if (repositoryResponse.statusCode() == 200) {
log.info("User has access to the repository.");
log.trace("Response: " + repositoryResponse.body());
return true
} else {
log.info("User does not have access to the repository " + repositoryResponse.statusCode());
return false
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@ interface RegistryLookupService {
* or {@code null} if nothing is found
*/
RegistryInfo lookup(String registry)

URI registryEndpoint(String registry)
}
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ class RegistryLookupServiceImpl implements RegistryLookupService {
* @param registry The registry name e.g. quay.io. When empty defaults to 'docker.io'
* @return the corresponding registry endpoint uri
*/
protected URI registryEndpoint(String registry) {
URI registryEndpoint(String registry) {
def result = registry ?: DOCKER_IO
if( result==DOCKER_IO )
result = DOCKER_REGISTRY_1
Expand Down
6 changes: 6 additions & 0 deletions src/main/groovy/io/seqera/wave/auth/RepositoryInfo.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package io.seqera.wave.auth

class RepositoryInfo {
String registry
String repository
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import javax.validation.Valid
import io.micronaut.http.annotation.Controller
import io.micronaut.http.annotation.Post
import io.seqera.wave.auth.RegistryAuthService
import io.seqera.wave.model.ValidateRegistryCredsRequest
import jakarta.inject.Inject
import reactor.core.publisher.Mono

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package io.seqera.wave.controller
package io.seqera.wave.model

import javax.annotation.Nullable
import javax.validation.constraints.NotBlank

import io.micronaut.core.annotation.Introspected
Expand Down
45 changes: 45 additions & 0 deletions src/test/groovy/io/seqera/wave/auth/RegistryLoginTest.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
package io.seqera.wave.auth

import spock.lang.Specification

import io.micronaut.test.extensions.spock.annotation.MicronautTest
import jakarta.inject.Inject

@MicronautTest
class RegistryLoginTest extends Specification{
@Inject
RegistryAuthServiceImpl impl

void 'test login with registry'() {
when:
def login = impl.login("docker.io","wavetest","dckr_pat_sShAQOWshE-y3SeE8wll774CWzM")

then:
login
}
void 'test valid login with repository'() {
when:
def login = impl.login("https://docker.io/hrma017/dev","hrma017","dckr_pat_NtfDznNlQjarjit3df4L713undw")
then:
login
}
void 'test invalid login with repository'() {
when:
def login = impl.login("docker.io/hrma017/dev","wavetest","dckr_pat_sShAQOWshE-y3SeE8wll774CWzM")
then:
!login
}
void 'test repository parser'(){
when:
def result = impl.parseURI("localhost")
then:
result.repository == null
}
void 'test repository parser'(){
when:
def result = impl.parseURI("https://docker.io/hrma017/dev")
then:
result.registry == "docker.io"
result.repository == "hrma017/dev"
}
}