Class MediaTexture

This class extends the Texture class specifically to handle external sources such as images, videos or canvases. It can be used with ComputePass and/or any kind of Mesh.

Can also handle texture transformations using a Mat3 if the useTransform parameter has been set to true upon creation.

If you use transformations, the modelMatrix will be available in the shaders using texturesMatrices.${texture.options.name}.matrix.

The library provide a convenient helpers in the shaders to help you compute the transformed UV:

// assuming 'uv' is a valid vec2f containing the original UV and the texture name is 'meshTexture'
uv = getUVCover(uv, texturesMatrices.meshTexture.matrix);

Example

// assuming 'renderer' is a valid GPURenderer

// create a simple media texture
const mediaTexture = new MediaTexture(renderer, {
  label: 'Media texture',
  name: 'mediaTexture',
})

mediaTexture.loadImage('path/to/image.jpg')

// create a cube map texture
const cubeMapTexture = new MediaTexture(renderer, {
  label: 'Cube map texture',
  name: 'cubeMapTexture',
  viewDimension: 'cube',
})

cubeMapTexture.loadImages([
  'path/to/positive-x.jpg',
  'path/to/negative-x.jpg',
  'path/to/positive-y.jpg',
  'path/to/negative-y.jpg',
  'path/to/positive-z.jpg',
  'path/to/negative-z.jpg',
])

Hierarchy (View Summary)

Texture
- MediaTexture
  - DOMTexture

Constructors

constructor

new MediaTexture(
renderer: Renderer | GPUCurtains,
parameters?: MediaTextureParams,
): MediaTexture
Texture constructor
Parameters
- renderer: Renderer | GPUCurtains
  renderer object or GPUCurtains class object used to create this Texture.
- parameters: MediaTextureParams = defaultMediaTextureParams
  parameters used to create this Texture.
Returns MediaTexture
Overrides Texture.constructor
- Defined in src/core/textures/MediaTexture.ts:160

Properties

externalTexture

externalTexture: GPUExternalTexture

The GPUExternalTexture used if any.

sources

sources: MediaTextureSource[]

The MediaTexture sources to use if any.

size

size: TextureSize

Size of the texture source, usually our sources first element size (since for cube maps, all sources must have the same size).

options

options: MediaTextureOptions

Options used to create this MediaTexture.

videoFrameCallbackIds

videoFrameCallbackIds: Map<number, number>

Array of requestVideoFrameCallback returned ids if used.

offset

offset: Vec2

Vec2 offset to apply to the Texture if useTransform parameter has been set to true.

scale

scale: Vec2

Vec2 scale to apply to the Texture if useTransform parameter has been set to true.

transformOrigin

transformOrigin: Vec2

Vec2 transformation origin to use when applying the transformations to the Texture if useTransform parameter has been set to true. A value of (0.5, 0.5) corresponds to the center of the texture. Default is (0, 0), the upper left.

modelMatrix

modelMatrix: Mat3

Mat3 transformation matrix to apply to the Texture if useTransform parameter has been set to true.

`Optional`transformBinding

transformBinding?: BufferBinding

BufferBinding to send the transformation matrix to the shaders if useTransform parameter has been set to true.

renderer

renderer: Renderer

renderer used by this Texture.

type

type: string

The type of the Texture.

`Readonly`uuid

uuid: string

The universal unique id of this Texture.

texture

texture: GPUTexture

The GPUTexture used.

bindings

bindings: BindGroupBindingElement[]

Array of bindings that will actually only hold one texture binding.

Accessors

sourcesLoaded

get sourcesLoaded(): boolean
Get whether all our sources have been loaded.

Returns boolean
- Defined in src/core/textures/MediaTexture.ts:238
set sourcesLoaded(value: boolean): void
Set whether all our sources have been loaded.
Parameters
- value: boolean
  boolean flag indicating if all the sources have been loaded.
Returns void
- Defined in src/core/textures/MediaTexture.ts:246

sourcesUploaded

get sourcesUploaded(): boolean
Get whether all our sources have been uploaded.

Returns boolean
- Defined in src/core/textures/MediaTexture.ts:257
set sourcesUploaded(value: boolean): void
Set whether all our sources have been uploaded.
Parameters
- value: boolean
  boolean flag indicating if all the sources have been uploaded
Returns void
- Defined in src/core/textures/MediaTexture.ts:265

rotation

get rotation(): number
Get the actual rotation value.

Returns number
- the actual rotation value.
- Defined in src/core/textures/MediaTexture.ts:278
set rotation(value: number): void
Set the actual rotation value and update the modelMatrix.
Parameters
- value: number
  new rotation value to use.
Returns void
- Defined in src/core/textures/MediaTexture.ts:286

textureBinding

get textureBinding(): TextureBinding
Get our texture binding.

Returns TextureBinding
Inherited from Texture.textureBinding
- Defined in src/core/textures/Texture.ts:202

Methods

_onSourceLoadedCallback

_onSourceLoadedCallback(source: TextureSource): void
function assigned to the onSourceLoaded callback
Parameters
- source: TextureSource
Returns void
- Defined in src/core/textures/MediaTexture.ts:136

_onAllSourcesLoadedCallback

_onAllSourcesLoadedCallback(): void
function assigned to the onAllSourcesLoaded callback

Returns void
- Defined in src/core/textures/MediaTexture.ts:141

_onSourceUploadedCallback

_onSourceUploadedCallback(source: TextureSource): void
function assigned to the onSourceUploaded callback
Parameters
- source: TextureSource
Returns void
- Defined in src/core/textures/MediaTexture.ts:146

_onAllSourcesUploadedCallback

_onAllSourcesUploadedCallback(): void
function assigned to the onAllSourcesUploaded callback

Returns void
- Defined in src/core/textures/MediaTexture.ts:151

updateModelMatrix

updateModelMatrix(): void
Update the modelMatrix using the offset, rotation, scale and transformOrigin and tell the transformBinding to update, only if useTransform parameter has been set to true.

Returns void
- Defined in src/core/textures/MediaTexture.ts:294

setBindings

setBindings(): void
Set our bindings.

Returns void
Overrides Texture.setBindings
- Defined in src/core/textures/MediaTexture.ts:317

copy

copy(texture: DOMTexture | MediaTexture | Texture): void
Copy another Texture into this Texture.
Parameters
- texture: DOMTexture | MediaTexture | Texture
  Texture to copy.
Returns void
Overrides Texture.copy
- Defined in src/core/textures/MediaTexture.ts:336

createTexture

createTexture(): void
Create the texture (or copy it from source) and update the binding resource.

Returns void
Overrides Texture.createTexture
- Defined in src/core/textures/MediaTexture.ts:370

uploadVideoTexture

uploadVideoTexture(): void
Import a GPUExternalTexture from the Renderer, update the textureBinding and its bind group

Returns void
- Defined in src/core/textures/MediaTexture.ts:421

setSourceSize

setSourceSize(): void
Set the size based on the first available loaded sources.

Returns void
- Defined in src/core/textures/MediaTexture.ts:436

loadImageBitmap

loadImageBitmap(url: string): Promise<ImageBitmap>
Load an HTMLImageElement from a URL and create an ImageBitmap to use as a source.
Parameters
- url: string
  URL of the image to load.
Returns Promise<ImageBitmap>
- the newly created ImageBitmap.
- Defined in src/core/textures/MediaTexture.ts:463

loadImage

loadImage(source: string | HTMLImageElement): Promise<void>
Load and create an ImageBitmap from a URL or HTMLImageElement, use it as a source and create the GPUTexture.
Parameters
- source: string | HTMLImageElement
  the image URL or HTMLImageElement to load.
Returns Promise<void>
- Defined in src/core/textures/MediaTexture.ts:487

useImageBitmap

useImageBitmap(imageBitmap: ImageBitmap, sourceIndex?: number): void
Use an already loaded ImageBitmap as a sources.
Parameters
- imageBitmap: ImageBitmap
  ImageBitmap to use.
- sourceIndex: number = 0
  Index at which to insert the source in the sources array in case of cube map.
Returns void
- Defined in src/core/textures/MediaTexture.ts:527

loadImages

loadImages(sources: (string | HTMLImageElement)[]): Promise<void>
Load and create images using loadImage from an array of images sources as strings or HTMLImageElement. Useful for cube maps.
Parameters
- sources: (string | HTMLImageElement)[]
  Array of images sources as strings or HTMLImageElement to load.
Returns Promise<void>
- Defined in src/core/textures/MediaTexture.ts:554

onVideoFrameCallback

onVideoFrameCallback(sourceIndex?: number): void
Set our source shouldUpdate flag to true at each new video frame.
Parameters
- sourceIndex: number = 0
Returns void
- Defined in src/core/textures/MediaTexture.ts:569

onVideoLoaded

onVideoLoaded(video: HTMLVideoElement, sourceIndex?: number): void
Callback to run when a HTMLVideoElement has loaded (when it has enough data to play). Set the HTMLVideoElement as a MediaTextureSource.source and create the GPUTexture or GPUExternalTexture.
Parameters
- video: HTMLVideoElement
  the newly loaded HTMLVideoElement.
- sourceIndex: number = 0
  Index of the HTMLVideoElement in the sources array.
Returns void
- Defined in src/core/textures/MediaTexture.ts:584

isVideoSource

isVideoSource(source: TextureSource): source is HTMLVideoElement
Get whether the provided source is a video.
Parameters
- source: TextureSource
  TextureSource to check.
Returns source is HTMLVideoElement
- Whether the source is a video or not.
- Defined in src/core/textures/MediaTexture.ts:614

isVideoSourceReady

isVideoSourceReady(source: TextureSource): boolean
Get whether the provided video source is ready to be played.
Parameters
- source: TextureSource
  TextureSource to check.
Returns boolean
- Whether the video source is ready to be played.
- Defined in src/core/textures/MediaTexture.ts:623

shouldUpdateVideoSource

shouldUpdateVideoSource(source: TextureSource): boolean
Get whether the provided video source is ready to be uploaded.
Parameters
- source: TextureSource
  TextureSource to check.
Returns boolean
- Whether the video source is ready to be uploaded.
- Defined in src/core/textures/MediaTexture.ts:634

loadVideo

loadVideo(source: string | HTMLVideoElement): void
Load a video from a URL or HTMLVideoElement and register onVideoLoaded callback.
Parameters
- source: string | HTMLVideoElement
  the video URL or HTMLVideoElement to load.
Returns void
- Defined in src/core/textures/MediaTexture.ts:644

loadVideos

loadVideos(sources: (string | HTMLVideoElement)[]): void
Load and create videos using loadVideo from an array of videos sources as strings or HTMLVideoElement. Useful for cube maps.
Parameters
- sources: (string | HTMLVideoElement)[]
  Array of images sources as strings or HTMLVideoElement to load.
Returns void
- Defined in src/core/textures/MediaTexture.ts:709

loadCanvas

loadCanvas(source: HTMLCanvasElement): void
Load a HTMLCanvasElement and use it as one of our sources.
Parameters
- source: HTMLCanvasElement
  the HTMLCanvasElement to use.
Returns void
- Defined in src/core/textures/MediaTexture.ts:719

loadCanvases

loadCanvases(sources: HTMLCanvasElement[]): void
Load an array of HTMLCanvasElement using loadCanvas . Useful for cube maps.
Parameters
- sources: HTMLCanvasElement[]
  Array of HTMLCanvasElement to load.
Returns void
- Defined in src/core/textures/MediaTexture.ts:753

setSourceUploaded

setSourceUploaded(sourceIndex?: number): void
Set the sourceUploaded flag to true for the source at a given index in our sources array. If all sources have been uploaded, set our sourcesUploaded flag to true.
Parameters
- sourceIndex: number = 0
  Index of the source in the sources array.
Returns void
- Defined in src/core/textures/MediaTexture.ts:780

onSourceLoaded

onSourceLoaded(callback: (source: TextureSource) => void): this
Callback to run when one of the source has been loaded.
Parameters
- callback: (source: TextureSource) => void
  callback to run when one of the source has been loaded.
Returns this
- our MediaTexture
- Defined in src/core/textures/MediaTexture.ts:796

onAllSourcesLoaded

onAllSourcesLoaded(callback: () => void): this
Callback to run when all of the source have been loaded.
Parameters
- callback: () => void
  callback to run when all of the sources have been loaded.
Returns this
- our MediaTexture
- Defined in src/core/textures/MediaTexture.ts:809

onSourceUploaded

onSourceUploaded(callback: (source: TextureSource) => void): this
Callback to run when one of the MediaTextureSource.source has been uploaded to the GPU.
Parameters
- callback: (source: TextureSource) => void
  callback to run when one of the source has been uploaded to the GPU.
Returns this
- our MediaTexture.
- Defined in src/core/textures/MediaTexture.ts:822

onAllSourcesUploaded

onAllSourcesUploaded(callback: () => void): this
Callback to run when all of the source have been uploaded to the GPU.
Parameters
- callback: () => void
  callback to run when all of the sources been uploaded to the GPU.
Returns this
- our MediaTexture.
- Defined in src/core/textures/MediaTexture.ts:835

update

update(): void
Update a MediaTexture by uploading the texture if needed.

Returns void
- Defined in src/core/textures/MediaTexture.ts:848

destroy

destroy(): void
Destroy the MediaTexture.

Returns void
Overrides Texture.destroy
- Defined in src/core/textures/MediaTexture.ts:889

setRenderer

setRenderer(renderer: Renderer | GPUCurtains): void
Reset this Texture renderer, and resize it if needed.
Parameters
- renderer: Renderer | GPUCurtains
  New Renderer or GPUCurtains instance to use.
Returns void
Inherited from Texture.setRenderer
- Defined in src/core/textures/Texture.ts:161

copyGPUTexture

copyGPUTexture(texture: GPUTexture): void
Copy a GPUTexture directly into this Texture. Mainly used for depth textures.
Parameters
- texture: GPUTexture
  GPUTexture to copy.
Returns void
Inherited from Texture.copyGPUTexture
- Defined in src/core/textures/Texture.ts:219

uploadSource

uploadSource(
    parameters: {
        source: GPUCopyExternalImageSource;
        width?: number;
        height?: number;
        depth?: number;
        origin?: GPUOrigin3D;
        colorSpace?: PredefinedColorSpace;
    },
): void
Upload a source to the GPU and use it for our texture.
Parameters
- parameters: {
      source: GPUCopyExternalImageSource;
      width?: number;
      height?: number;
      depth?: number;
      origin?: GPUOrigin3D;
      colorSpace?: PredefinedColorSpace;
  }
  parameters used to upload the source.
  - source: GPUCopyExternalImageSource
    source to use for our texture.
  - Optionalwidth?: number
    source width.
  - Optionalheight?: number
    source height.
  - Optionaldepth?: number
    source depth.
  - Optionalorigin?: GPUOrigin3D
    ().destination.origin | GPUOrigin3D of the source copy.
  - OptionalcolorSpace?: PredefinedColorSpace
Returns void
Inherited from Texture.uploadSource
- Defined in src/core/textures/Texture.ts:276

uploadData

uploadData(
    parameters: {
        width?: number;
        height?: number;
        depth?: number;
        origin?: GPUOrigin3D;
        data?: Float32Array;
    },
): void
Use data as the texture source and upload it to the GPU.
Parameters
- parameters: {
      width?: number;
      height?: number;
      depth?: number;
      origin?: GPUOrigin3D;
      data?: Float32Array;
  }
  parameters used to upload the source.
  - Optionalwidth?: number
    data source width.
  - Optionalheight?: number
    data source height.
  - Optionaldepth?: number
    data source depth.
  - Optionalorigin?: GPUOrigin3D
    ().destination.origin | GPUOrigin3D of the data source copy.
  - Optionaldata?: Float32Array
    Float32Array data to use as source.
Returns void
Inherited from Texture.uploadData
- Defined in src/core/textures/Texture.ts:311

resize

resize(size?: TextureSize): void
Resize our Texture, which means recreate it/copy it again and tell the texture bind group to update.
Parameters
- size: TextureSize = null
  the optional new size to set.
Returns void
Inherited from Texture.resize
- Defined in src/core/textures/Texture.ts:340

Class MediaTexture

Example

Hierarchy (View Summary)

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

Returns MediaTexture

Properties

externalTexture

sources

size

options

videoFrameCallbackIds

offset

scale

transformOrigin

modelMatrix

OptionaltransformBinding

renderer

type

Readonlyuuid

texture

bindings

Accessors

sourcesLoaded

Returns boolean

Parameters

Returns void

sourcesUploaded

Returns boolean

Parameters

Returns void

rotation

Returns number

Parameters

Returns void

textureBinding

Returns TextureBinding

Methods

_onSourceLoadedCallback

Parameters

Returns void

_onAllSourcesLoadedCallback

Returns void

_onSourceUploadedCallback

Parameters

Returns void

_onAllSourcesUploadedCallback

Returns void

updateModelMatrix

Returns void

setBindings

Returns void

copy

Parameters

Returns void

createTexture

Returns void

uploadVideoTexture

Returns void

setSourceSize

Returns void

loadImageBitmap

Parameters

Returns Promise<ImageBitmap>

loadImage

Parameters

Returns Promise<void>

useImageBitmap

Parameters

Returns void

loadImages

Parameters

Returns Promise<void>

onVideoFrameCallback

`Optional`transformBinding

`Readonly`uuid

`Optional`width?: number

`Optional`height?: number

`Optional`depth?: number

`Optional`origin?: GPUOrigin3D

`Optional`colorSpace?: PredefinedColorSpace

`Optional`width?: number

`Optional`height?: number

`Optional`depth?: number

`Optional`origin?: GPUOrigin3D

`Optional`data?: Float32Array