You need to use the Typescript type assertion feature.
In your case the compiler complains because myObs$ must be of type Observable<A> while your custom operator can return either an Observable<A> or an Observable<B>.
So if you want to avoid the error you need to reassure Typescript that your custom operator will certainly return an Observable<A>. This is a trick to avoid a legitimate check of the Typescript compiler, so you better be sure you are doing the right thing.
So the code should look like this
myObs$: Observable<A> = this.mySub.asObservable().pipe(this.customOperator) as Observable<A>;
or, more simply
myObs$ = this.mySub.asObservable().pipe(this.customOperator) as Observable<A>;
If you hover over myObs$, for instance within VSCode, you will see that the type inferred for myObs$ is actually Observable<A> even if you have not specified the type (at least in this last version of the code)
UPDATE based on @akotech response
A better way to approach this case is what has been proposed by @akotech.
customOperator as coded below returns an Observable<U>
private customOperator<T extends IModel<U>, U extends A | B> () {
return (custom: Observable<T>): Observable<U> =>
custom.pipe(map((x: T) => x.value ));
}
Interestingly, at least in my case, if I do not declare the type of myObs$ like in this line
myObs$ = mySub.pipe(customOperator_())
the type inferred for myObs$ is Observable<A | B> but, at the same time, if I try something like this
myObs$: : Observable<B> = mySub.pipe(customOperator_())
I get an error from the compiler Type 'Observable<IModel>' is not assignable to type 'Observable<IModel>', which is correct since mySub notifies objects of type IModel<A>.
At the same time, if I try something like this
myObs$: : Observable<A> = mySub_A.pipe(customOperator_())
no error is raised by the compiler, which again is correct.
So the solution proposed by @akotech is safer than the one proposed by me.