How are static assets handled?

[javiereguiluz]

A question by @jrobeson on Symfony Slack:

So, with the manifest file approach for versioning, how are static assets handled? like referenced images that aren't in CSS.

I was previously using the hashed-asset-bundle, but you can't use the new manifest versioning with it.

[ghost]

Yes, you can't currently use json_manifest_path alongside a normal symfony version_strategy.

[stof]

@jrobeson actually you can. Symfony supports having several asset packages with different versioning strategies. so you could have 1 package using the manifest strategy, and another one using hashed-asset-bundle (or just use hashed-asset-bundle for all of them, by disabling the path-based versioning of encore)

[ghost]

Oh packages! I had forgotten that they exist. I guess that's one way to go about it.

[ghost]

I wonder if other folks are gonna use different versioning schemes for these static assets vs webpack or just me. If so, it might be good to see a recommendation somewhere.

[BPScott]

+1 for a mechanism to add static assets such as images into the manifest.json.

Currently I have a gulp file generates a manifest that deals with hashing my css, js and images, if I were to use encore I'd like to use the same versioning strategy for all my static assets js and css.

[weaverryan]

+1 for there being a way for processing these through webpack. Actually, it's pretty standard already - if you (for example) require a image or font from CSS (or even JS), it will be moved into the build/ directory and added to the manifest. But, I want to make sure we're following whatever the best "best-practice" is for this. There are also simple "copy" plugins, but I don't like this solution - afaik, those assets don't go through webpack, so they're not added to the manifest.

[davidmpaz]

The main problem for me here is that templates (Twig or else) contain references to assets (images) that will never be handled by webpack unless we do it explicitely.

Solutions tried were:

1. At build time try to make webpack require all resources in a directory. This did not work as expected. I still need a file in my application modules doing that logic. I dont want build logic mixed with my application.

2. Use copy plugin. This does what I need with the drawback exposed before about the manifest. Still there are request for adding support to it in [Feature] Support manifest file webpack-contrib/copy-webpack-plugin#104 is not active I would say.

So far I havent found anything else. If somebody has ideas pls share :)

[skaryys]

@davidmpaz Is there now any working solution? I am using Encore now together with CopyWebpackPlugin and ImageMinPlugin and it works -> but with one problem. I am not able to put copied images to manifest.json.
I tried to use your last pull request: https://github.com/danethurber/webpack-manifest-plugin/pull/45, but it does not generate files with hash in filename.

If it can be done now, I would really appreciate an example. Thanks!

[weaverryan]

There's no working solution yet - but it's on my "todo" list.

As a workaround, you should be able to add a require call in any .js file for the assets you need. For example:

# app.js

require('./images/foo.png');

This should cause foo.png to be copied into the build directory and be in the manifest.json file. As an added bonus, when you require a static file like this, the require() call actually returns the final path to the asset, so you can use it in js if needed :).

Let me know if that works. I'd like a more official way to do this, but really, this is what that official solution will be doing behind the scenes.

Cheers!

[skaryys]

It works, thanks!
I think i can live with that now. In project with many images, it could be problem to require them all like this, but thats not my case.

[davidmpaz]

@skaryys your scenario is the same as mine. For me is not good also to require all images since it is too much and at that point that is a build/deploy task, I did not want it in application code.

did you read about plugin execution order? This thread gave me the hints needed.

Basically unshift your plugin so it executes before the manifest plugin.

My working solution with PR applied, not using it yet though :)

let CopyWebpackPlugin = require('copy-webpack-plugin');
        // important to use `unshift` since this plugin should execute first
        // or at least before ManifestPlugin to get manifest right
        webpackConfig.plugins.unshift(new CopyWebpackPlugin([
            {
                from: resourcesPath + '/img',
                to: outputPath + '/images'
            }
        ]));

Hints for the future:
Try to make referenced links for issues threads ;) ... Remember that references to other issues or pull requests are important since can make more visible the reach of that issue. It is better to have something like shellscape/webpack-manifest-plugin#45 instead the full url like https://github.com/danethurber/webpack-manifest-plugin/pull/45, the first create a reference in that other thread, the second doeasn't. Syntax in github markdown danethurber/webpack-manifest-plugin#45

let me know if this helps.

[skaryys]

@davidmpaz Ok, i was able to generate manifest.json file now thanks to you including the copied assets. Have you somehow managed to copy statics files with adding hash to their filenames?
I am now using temporary solution from @weaverryan, but in future in osme project it would be really annoying require all the images as you had said before.

Thanks anyway for the unshift solution and hints ;)

[davidmpaz]

I am glad it helped ;)....

Copy plugin has some functionality for naming files with hashes and so on but you need to try.

I had problem with that part, somehow the copy plugin was messing around when I passed some globs like: img/**/* you need to try yourself, please report results if possible.

I tried:
Using simple names, for example not recursing inside dir, plugin worked fine for adding hashes like:
img/[name].[hash].[ext] but like this it was missing all other subdirs as far I remember.

I did not dive into it since in my case those images copied will rarely change, so it can just be copied as is. I don't need hashing them. With having it in manifest to integrate with framework later on was good enough.

[skaryys]

@davidmpaz I was succesfully able to copy assets with hash but the manifest then looked like this:

{
"/img/image-[hash].jpg": "/img/image-[hash].jpg"
}

So instead of dealing with the ManifestPlugin, I wrote little script which generates javascript file which looks like this:

const image = require(path_to_image);
const anotherimage = require(path_to_image);

For me and my project, it works. I am quite unskilled with NodeJS and creating packages, but if it will be useful for someone, I will be glad ;)

skaryys/encore-require-assets-helper

[Sydney-o9]

For those of you who struggle with what is mentioned above.

1. Require your image in your js file

assets/js/app.js

require('./images/foo.png');

Your image will be generated in your public/build/images directory:

# We are in 'public/images'
$ tree
├── images
│   ├── foo.fe373bcc.png
├── main
│   └── app.js
└── manifest.json

2. Compile assets

$ ./node_modules/.bin/encore dev

Among other things, your manifest.json has been generated.

{
  "build/images/foo.png": "/build/images/foo.fe373bcc.png",
  "build/main/app.js": "/build/main/app.js"
}

3. Don't forget to use json_manifest:

packages/framework.yaml

framework
    assets:
        json_manifest_path: '%kernel.project_dir%/public/build/manifest.json'

[bravik]

@weaverryan
Could you please make a small clarification.
When I require images like you've advised, but adding a subfolder:
require('../images/homepage/image.jpg');
the actual file is copied to: assets/images/image.<hash>.jpg
and not to assets/images/homepage/image.<hash>.jpg as I would expect.
(assets here is a build dir in public folder)

So I can not organize images on subfolders. Why first level folder images is been somehow detected and created but not subfoldesr? Is this a correct behaviour? If so are there any docs to read about how this works?

[Lyrkan]

@bravik By default all images are put into <outputPath>/images:

webpack-encore/lib/config-generator.js

Lines 143 to 156 in 00e4051

	// Default filename can be overriden using Encore.configureFilenames({ images: '...' })
	let filename = 'images/[name].[hash:8].[ext]';
	if (this.webpackConfig.configuredFilenames.images) {
	filename = this.webpackConfig.configuredFilenames.images;
	}
	rules.push({
	test: /\.(png|jpg|jpeg|gif|ico|svg|webp)$/,
	loader: 'file-loader',
	options: {
	name: filename,
	publicPath: this.webpackConfig.getRealPublicPath()
	}
	});

If you want to keep your subfolders you'll have to change the naming strategy to include the [path] placeholder. You can do that using the configureFilenames method, for instance:

Encore.configureFilenames({
    images: '[path][name].[hash:8].[ext]'
});

[kaizokou]

All this is really confusing ...

To enable the versioning and have the manifest file working I followed this :

https://symfony.com/doc/current/frontend/encore/versioning.html

The only difference is that I'm working with the dev environment so I needed to change

.enableVersioning(Encore.isProduction()) to .enableVersioning()

in the default webpack.config.js file.

Also to make it work I needed to add this key under assets: in framework.yaml

base_urls: - 'http://local.myproject.com/public/'

it's really ugly but the asset fonction don't put the /public in the url

[Lyrkan]

Hi @kaizokou,

Also to make it work I needed to add this key under assets: in framework.yaml
base_urls: - 'http://local.myproject.com/public/'
it's really ugly but the asset fonction don't put the /public in the url

Which version of Symfony are you using? Starting from Symfony 4 the web directory (where you put your publicly accessible files) has been renamed public, so you shouldn't need to include it in your URLs.

If you use an older version of Symfony you should use Encore.setOutputPath('web/build/') and not Encore.setOutputPath('public/build/') as described on the following page: https://symfony.com/doc/3.4/frontend/encore/simple-example.html#configuring-encore-webpack

[kaizokou]

@Lyrkan I'm using SF4 latest version and I don't have web directory. It's a new project without much things in it. Maybe I'm missing something in the config.

The asset twig fonction was working well before enabling the versioning.

Maybe this is because I'm using Mamp and the web server root is on / and not on /public. I do have a .htaccess that redirect to /public. I don't know.

[Lyrkan]

@kaizokou Indeed, if your Symfony app is in a subdirectory (which is kind of the same thing) you'll have to do that kind of adjustment (you should be able to tweak MAMP to avoid it though, for instance by creating vhosts).

Did you try using framework.assets.base_path instead? It may be "cleaner" than framework.assets.base_urls for your use-case.

By the way (and not really related to Encore), if you generate URLs from the console you'll also need to set router.request_context.base_url so it knows about that subfolder.

[kaizokou]

At the end I deactivated completely the versioning because I don't need it for the images.
At the moment I put all my images inside /public/img directly.

Originally I put them under /assets/img because I thought this is where they should live. But then it doesn't seem to be easily copied by Webpack inside the /public/build repository.
Moreover I'm not sure I want Webpack to handle this task as the images never change, so it's a lot of extra work for nothing.

That said, it bothers me to have my asset in two places : /assets for css/js/font and /public/img for images. Plus, I also have an /assets/img folder for graphics elements included in the css (svg ...).

The Symfony Best Practices pdf is quite poor about the assets. The chapter 10 is only 1 page and doesn't say anything except that Encore is the recommended asset manager.

At the end I'm really confused about how to handle my assets.

I'll be so grateful if someone can clear this out.

btw thanks @Lyrkan for your answer.

[etshy]

Hi
Have some questions about image assets again.
Is there a way to include all the images from a folder and subfolder ?
I have my image in assets/img/* and my main.js (where I do the require) in assets/js/main.js
and I would like to do a thing like require('../img/*') to include all images.
require all images one by one is not really convenient.

[Lyrkan]

Hey @etshy,

You could probably achieve that using require.context.

For instance:

const imagesCtx = require.context('./images', false, /\.(png|jpg|jpeg|gif|ico|svg|webp)$/);
imagesCtx.keys().forEach(imagesCtx);

If you also want to include subfolders, change the second parameter of the require.context call to true.

A cleaner solution for that use case would probably be to use the CopyWebpackPlugin but there is an issue at the moment that prevents copied files from appearing in the manifest.json file.

[etshy]

thanks, seems it's working.
I tried to use plugin like require-all to require a folder but i didn't make it work cas the fs module was empty, so I don't know if this couls work too.

Thanks anyways for that solution, I'm pretty new with webpack and npm (I just used it to get some library before)

edit: hmm I have a little problem about output folder.
The images are put in a assets folder like public/build/assets/img instead of public/build/images
@Lyrkan do you know why ?
I used

.configureFilenames({
        images: '[path][name].[hash:8].[ext]'
})

and the piece of codes you put earlier.
I wanted to keep the folder hierarchy in my assets/img folder.

[Lyrkan]

@etshy That's because of how [path] works.

Before calling configureFilenames the old value for images filenames was images/[name].[hash:8].[ext], so all of them were put into your public/build/images folder.

Now let's say you use [name].[hash:8].[ext] (without the images/ part): all images would be generated directly into public/build.

Then, if you add [path] at the beginning of it you're basically asking the file-loader "keep the structure used by my original files". So for instance if your images are in the assets/img directory you get public/build/assets/img.

If you want to remove the assets/img part you'll have to do more changes directly in the generated Webpack conf and use the context options of the file-loader.

In case that helps, I did some tests a while ago that show how it behaves using different parameters: #103 (comment)

[etshy]

hmm Like I said I'm very new to this so I'm a bit lost.
where do I have to use the context option and how to remove the assets/img part ?
is it the require.context() you talked about before ?
I used this require.contect() in assets/js/main.js and here is the code I did

//require all images
const imagesCtx = require.context('../img', true, /\.(png|jpg|jpeg|gif|ico|svg|webp)$/);
//dunno what this line is doing exactly
imagesCtx.keys().forEach(imagesCtx);

Here is my webpack.config.js file if that could help you to guide me.

// webpack.config.js
var Encore = require('@symfony/webpack-encore');

Encore
// the project directory where all compiled assets will be stored
    .setOutputPath('public/build/')

    // the public path used by the web server to access the previous directory
    .setPublicPath('/build')

    // will create public/build/main.js and public/build/main.css
    .addEntry('main', './assets/js/main.js')

    .addEntry('reader', './assets/js/reader.js')

    // allow sass/scss files to be processed
    .enableSassLoader()

    // allow legacy applications to use $/jQuery as a global variable
    .autoProvidejQuery()

    .enableSourceMaps(!Encore.isProduction())

    // empty the outputPath dir before each build
    .cleanupOutputBeforeBuild()

    // show OS notifications when builds finish/fail
    .enableBuildNotifications()

// create hashed filenames (e.g. app.abc123.css)
    .enableVersioning()

    .configureFilenames({
        images: '[path][name].[hash:8].[ext]'
    })
;

// export the final configuration
module.exports = Encore.getWebpackConfig();

[Lyrkan]

The context option I'm talking about is totally unrelated to the require.context call from before.

And as how to do it... that's not something that you can do by only using Encore methods (that's what I meant by "you'll have to do more changes directly in the generated Webpack conf").

You'd have to:

• retrieve the Webpack config (= the result of getWebpackConfig())
• edit it manually (retrieve the file-loader instance that handles images and change its options to set that context)
• export the updated config

Do you really want to do that though? You shouldn't have to worry about the directories your images are put into. Especially since only changing that loader's context to assets/img will probably have some side-effects if you (or one of the libs you use) require images from another folder.

[etshy]

hm yeah that's too much just to have the folder hierarchy I want.

I thought it was weird to have build/assets/img/* as folder so wanted to fix that and I do't think generate all image in the same folders is good but...

What's the recommended ways to handle images so ?
export all image in public/build/images and have them all mixed, or have suplementary folders "for nothing" like public/build/assets/*yourImageFolderHere* ?

Because, I guess imags used in html/twig or whatever (not detected during the encore exportation) have to be manually get by url in the images folder, right ?

[gnorby88]

// main.js

const imagesContext = require.context('../img', true, /\.(png|jpg|jpeg|gif|ico|svg|webp)$/);
imagesContext.keys().forEach(imagesContext);

// webpack.config.js

var Encore = require('@symfony/webpack-encore');
Encore
    // your configs..

    // disable the default images loader.
    .disableImagesLoader()

    .addLoader({
        test: /\.(png|jpg|jpeg|gif|ico|svg)$/,
        use: [{
            loader: 'file-loader',
            options: {
                name: '[path][name].[hash:8].[ext]',
                context: './assets',
            }
        }]
    })

module.exports = Encore.getWebpackConfig();

[weaverryan]

There is now a copyFiles() plugin - it works great :)

[chadyred]

The documentation is update : https://symfony.com/doc/current/frontend/encore/copy-files.html