Skip to main content

ServingCost

logicaffeine_compile::spatial::serving

Struct ServingCost 

Source 
pub struct ServingCost {
Show 25 fields
    pub card: &'static str,
    pub batch: u32,
    pub host_sequences: u32,
    pub rows_per_sequence: u32,
    pub emitted_per_sequence: f64,
    pub weight_bytes: f64,
    pub sequence_bytes: f64,
    pub rewrite_bytes: f64,
    pub array_macs: f64,
    pub vector_macs: f64,
    pub scale_cycles: f64,
    pub memory_s: f64,
    pub host_s: f64,
    pub array_s: f64,
    pub vector_s: f64,
    pub scale_s: f64,
    pub pass_s: f64,
    pub binds: ServingBind,
    pub held_bytes: f64,
    pub capacity_bytes: f64,
    pub host_held_bytes: f64,
    pub host_capacity_bytes: f64,
    pub on_chip_bytes: f64,
    pub on_chip_capacity: f64,
    pub provenance: Provenance,

}
Expand description

One pass of a serving on a card.

Fields§

§card: &'static str§batch: u32§host_sequences: u32

Of the batch, the sequences whose state and caches live in host memory.

§rows_per_sequence: u32§emitted_per_sequence: f64§weight_bytes: f64

Weight bytes a pass streams, shared by the batch, embedding rows included.

§sequence_bytes: f64

Bytes a pass moves for each sequence: its state, its caches, its log, its conv ring.

§rewrite_bytes: f64

Of sequence_bytes, the recurrent states’ bytes read and written back in place.

§array_macs: f64§vector_macs: f64§scale_cycles: f64§memory_s: f64§host_s: f64§array_s: f64§vector_s: f64§scale_s: f64§pass_s: f64

The largest of the five: the units and the memories overlap, so no pass is faster.

§binds: ServingBind§held_bytes: f64§capacity_bytes: f64§host_held_bytes: f64§host_capacity_bytes: f64§on_chip_bytes: f64§on_chip_capacity: f64§provenance: Provenance

The weakest fact the price rests on.

Implementations§

Trait Implementations§

Source§

impl Clone for ServingCost

Source§

fn clone(&self) -> ServingCost

Returns a duplicate of the value. Read more
1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for ServingCost

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl PartialEq for ServingCost

Source§

fn eq(&self, other: &ServingCost) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 · Source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
Source§

impl Serialize for ServingCost

Source§

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>
where __S: Serializer,

Serialize this value into the given Serde serializer. Read more
Source§

impl StructuralPartialEq for ServingCost

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
§

impl<T> Downcast for T
where T: Any,

§

fn into_any(self: Box<T>) -> Box<dyn Any>

Convert Box<dyn Trait> (where Trait: Downcast) to Box<dyn Any>. Box<dyn Any> can then be further downcast into Box<ConcreteType> where ConcreteType implements Trait.
§

fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>

Convert Rc<Trait> (where Trait: Downcast) to Rc<Any>. Rc<Any> can then be further downcast into Rc<ConcreteType> where ConcreteType implements Trait.
§

fn as_any(&self) -> &(dyn Any + 'static)

Convert &Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot generate &Any’s vtable from &Trait’s.
§

fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)

Convert &mut Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot generate &mut Any’s vtable from &mut Trait’s.
§

impl<T> DowncastSync for T
where T: Any + Send + Sync,

§

fn into_any_arc(self: Arc<T>) -> Arc<dyn Any + Send + Sync>

Convert Arc<Trait> (where Trait: Downcast) to Arc<Any>. Arc<Any> can then be further downcast into Arc<ConcreteType> where ConcreteType implements Trait.
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,